Dynamic user interface in machine-to-machine systems

ABSTRACT

User inputs received through a graphical user interface of a programming tool are used to define a set of capability abstractions for a particular application and further define, based on the one or more user inputs, relationships between the set of capability abstractions. The particular application is to utilize a machine-to-machine network, and the set of capability abstractions include: a sensor capability abstraction, an actuator capability abstraction, a computation logic capability abstraction, an input user interface (UI) capability abstraction, and an output UI capability abstraction for the particular application. The relationships include a relationship between the input UI capability abstraction and the computation logic capability abstraction, where the input UI capability is to provide an input to the computation logic capability abstraction. Service logic for the particular application is generated to deploy an instance of the particular application in an environment based on the set of capability abstractions and relationships.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/306,571, filed on Dec. 1, 2018, and entitled DYNAMIC USER INTERFACEIN MACHINE-TO-MACHINE SYSTEMS, which is a national stage applicationunder 35 U.S.C. § 371 of PCT International Application Serial No.PCT/US2016/040627, filed on Jul. 1, 2016, and entitled DYNAMIC USERINTERFACE IN MACHINE-TO-MACHINE SYSTEMS. The disclosure of the priorapplications are considered part of and are hereby incorporated byreference in their entirety in the disclosure of this application.

TECHNICAL FIELD

This disclosure relates in general to the field of computer systems and,more particularly, to managing machine-to-machine systems.

BACKGROUND

The Internet has enabled interconnection of different computer networksall over the world. While previously, Internet-connectivity was limitedto conventional general purpose computing systems, ever increasingnumbers and types of products are being redesigned to accommodateconnectivity with other devices over computer networks, including theInternet. For example, smart phones, tablet computers, wearables, andother mobile computing devices have become very popular, evensupplanting larger, more traditional general purpose computing devices,such as traditional desktop computers in recent years. Increasingly,tasks traditionally performed on a general purpose computers areperformed using mobile computing devices with smaller form factors andmore constrained features sets and operating systems. Further,traditional appliances and devices are becoming “smarter” as they areubiquitous and equipped with functionality to connect to or consumecontent from the Internet. For instance, devices, such as televisions,gaming systems, household appliances, thermostats, automobiles, watches,have been outfitted with network adapters to allow the devices toconnect with the Internet (or another device) either directly or througha connection with another computer connected to the network.Additionally, this increasing universe of interconnected devices hasalso facilitated an increase in computer-controlled sensors that arelikewise interconnected and collecting new and large sets of data. Theinterconnection of an increasingly large number of devices, or “things,”is believed to foreshadow a new era of advanced automation andinterconnectivity, referred to, sometimes, as the Internet of Things(IoT).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates an embodiment of a system including multiple sensordevices and an example management system.

FIG. 1B illustrates an embodiment of a cloud computing network.

FIG. 2 illustrates an embodiment of a system including an examplemanagement system.

FIG. 3A is a simplified block diagram illustrating an example of assetabstraction and binding.

FIG. 3B is a simplified block diagram illustrating an example of assetdiscovery.

FIG. 3C is a simplified block diagram illustrating an example of assetabstraction and binding using a discovered set of assets.

FIGS. 4A-4B are block diagrams representing abstraction layers of anexample machine-to-machine application deployment.

FIG. 5A is a screenshot of an example graphical development tool forcreating new machine-to-machine services.

FIG. 5B is a simplified block diagram illustrating an example deploymentof a new machine-to-machine service.

FIG. 6 is a simplified block diagram illustrating an example of managingapplication of configuration settings between two instances of aparticular machine-to-machine system.

FIG. 7 is a simplified block diagram illustrating an example system usedto deploy a particular machine-to-machine system.

FIG. 8A is a flowchart illustrating an example technique for developingan application using asset abstraction.

FIG. 8B is a flowchart illustrating an example technique for deployingan example machine-to-machine network utilizing asset abstraction.

FIG. 9 is a block diagram of an exemplary processor in accordance withone embodiment; and

FIG. 10 is a block diagram of an exemplary computing system inaccordance with one embodiment.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

FIG. 1A is a block diagram illustrating a simplified representation of asystem 100 that includes one or more devices 105 a-d, or assets,deployed throughout an environment. Each device 105 a-d may include acomputer processor and/or communications module to allow each device 105a-d to interoperate with one or more other devices (e.g., 105 a-d) orsystems in the environment. Each device can further include one or moreinstances of various types of sensors (e.g., 110 a-c), actuators (e.g.,115 a-b), storage, power, computer processing, and communicationfunctionality which can be leveraged and utilized (e.g., by otherdevices or software) within a machine-to-machine, or Internet of Things(IoT) system or application. In some cases, inter-device communicationand even deployment of an IoT application may be facilitated by one ormore gateway devices through which one or more of the devices (e.g., 105a-d) communicate and gain access to other devices and systems in one ormore networks (e.g., 120).

Sensors, or sensor assets, are capable of detecting, measuring, andgenerating sensor data describing characteristics of the environment inwhich they reside, are mounted, or are in contact with. For instance, agiven sensor (e.g., 110 a-c) may be configured to detect one or morerespective characteristics such as movement, weight, physical contact,temperature, wind, noise, light, computer communications, wirelesssignals, position, humidity, the presence of radiation, liquid, orspecific chemical compounds, among several other examples. Indeed,sensors (e.g., 110 a-c) as described herein, anticipate the developmentof a potentially limitless universe of various sensors, each designed toand capable of detecting, and generating corresponding sensor data for,new and known environmental characteristics. Actuators (e.g., 115 a-b)can allow the device to perform some kind of action to affect itsenvironment. For instance, one or more of the devices (e.g., 105 b, d)may include one or more respective actuators that accepts an input andperform its respective action in response. Actuators can includecontrollers to activate additional functionality, such as an actuator toselectively toggle the power or operation of an alarm, camera (or othersensors), heating, ventilation, and air conditioning (HVAC) appliance,household appliance, in-vehicle device, lighting, among other examples.

In some implementations, sensors 110 a-c and actuators 115 a-b providedon devices 105 a-d can be assets incorporated in and/or forming anInternet of Things (IoT) or machine-to-machine (M2M) system. IoT systemscan refer to new or improved ad-hoc systems and networks composed ofmultiple different devices interoperating and synergizing to deliver oneor more results or deliverables. Such ad-hoc systems are emerging asmore and more products and equipment evolve to become “smart” in thatthey are controlled or monitored by computing processors and providedwith facilities to communicate, through computer-implemented mechanisms,with other computing devices (and products having network communicationcapabilities). For instance, IoT systems can include networks built fromsensors and communication modules integrated in or attached to “things”such as equipment, toys, tools, vehicles, etc. and even living things(e.g., plants, animals, humans, etc.). In some instances, an IoT systemcan develop organically or unexpectedly, with a collection of sensorsmonitoring a variety of things and related environments andinterconnecting with data analytics systems and/or systems controllingone or more other smart devices to enable various use cases andapplication, including previously unknown use cases. Further, IoTsystems can be formed from devices that hitherto had no contact witheach other, with the system being composed and automatically configuredspontaneously or on the fly (e.g., in accordance with an IoT applicationdefining or controlling the interactions). Further, IoT systems canoften be composed of a complex and diverse collection of connecteddevices (e.g., 105 a-d), such as devices sourced or controlled by variedgroups of entities and employing varied hardware, operating systems,software applications, and technologies. In some cases, a gateway (e.g.,150) may be provided to localize a particular IoT system, with thegateway able to detect nearby devices (e.g., 105 a-d) and deploy (e.g.,in an automated, impromptu manner) an instance of a particular IoTapplication by orchestrating configuration of these detected devices tosatisfy requirements of the particular IoT application, among otherexamples.

Facilitating the successful interoperability of such diverse systems is,among other example considerations, an important issue when building ordefining an IoT system. Software applications can be developed to governhow a collection of IoT devices can interact to achieve a particulargoal or service. In some cases, the IoT devices may not have beenoriginally built or intended to participate in such a service or incooperation with one or more other types of IoT devices. Indeed, part ofthe promise of the Internet of Things is that innovators in many fieldswill dream up new applications involving diverse groupings of the IoTdevices as such devices become more commonplace and new “smart” or“connected” devices emerge. However, the act of programming, or coding,such IoT applications may be unfamiliar to many of these potentialinnovators, thereby limiting the ability of these new applications to bedeveloped and come to market, among other examples and issues.

As shown in the example of FIG. 1A, multiple IoT devices (e.g., 105 a-d)can be provided from which one or more different IoT applications can bebuilt. For instance, a device (e.g., 105 a-d) can include such examplesas a mobile personal computing device, such as a smart phone or tabletdevice, a wearable computing device (e.g., a smart watch, smart garment,smart glasses, smart helmet, headset, etc.), purpose-built devices suchas and less conventional computer-enhanced products such as home,building, vehicle automation devices (e.g., smartheat-ventilation-air-conditioning (HVAC) controllers and sensors, lightdetection and controls, energy management tools, etc.), smart appliances(e.g., smart televisions, smart refrigerators, etc.), and otherexamples. Some devices can be purpose-built to host sensor and/oractuator resources, such as a weather sensor devices that includemultiple sensors related to weather monitoring (e.g., temperature, wind,humidity sensors, etc.), traffic sensors and controllers, among manyother examples. Some devices may be statically located, such as a devicemounted within a building, on a lamppost, sign, water tower, secured toa floor (e.g., indoor or outdoor), or other fixed or static structure.Other devices may be mobile, such as a sensor provisioned in theinterior or exterior of a vehicle, in-package sensors (e.g., fortracking cargo), wearable devices worn by active human or animal users,an aerial, ground-based, or underwater drone among other examples.Indeed, it may be desired that some sensors move within an environmentand applications can be built around use cases involving a movingsubject or changing environment using such devices, including use casesinvolving both moving and static devices, among other examples.

Continuing with the example of FIG. 1A, software-based IoT managementplatforms can be provided to allow developers and end users to build andconfigure IoT applications and systems. An IoT application can providesoftware support to organize and manage the operation of a set of IoTdevice for a particular purpose or use case. In some cases, an IoTapplication can be embodied as an application on an operating system ofa user computing device (e.g., 125), a mobile app for execution on asmart phone, tablet, smart watch, or other mobile device (e.g., 130,135), a remote server, and/or gateway device (e.g., 150). In some cases,the application can have or make use of an application managementutility allowing users to configure settings and policies to govern howthe set devices (e.g., 105 a-d) are to operate within the context of theapplication. A management utility can also be used to orchestrate thedeployment of a particular instance of an IoT application, including theautomated selection and configuration of devices (and their assets) thatare to be used with the application. In some cases, an IoT managementapplication may be provided (e.g., on a gateway, user device, orcloud-based server, etc.), which can manage potentially multipledifferent IoT applications or systems. Indeed, an IoT managementapplication, or system, may be hosted on a single system, such as asingle server system (e.g., 140), a single end-user device (e.g., 125,130, 135), or a single gateway device (e.g., 150), among other examples.Alternatively, an IoT management system can be distributed acrossmultiple hosting devices (e.g., 125, 130, 135, 140, 150, etc.).

As noted above, IoT applications may be localized, such that a serviceis implemented utilizing an IoT system (e.g., of devices 105 a-d) withina specific geographic area, room, or location. In some instances, IoTdevices (e.g., 105 a-d) may connect to one or more gateway devices(e.g., 150) on which a portion of management functionality (e.g., asshared with or supported by management system 140) and a portion ofapplication service functionality (e.g., as shared with or supported byapplication system 145). Service logic and configuration data may bepushed (or pulled) to the gateway device 150 and used to configure IoTdevices (e.g., 105 a-d, 130, 135, etc.) within range or proximity of thegateway device 150 to allow the set of devices to implement a particularservice within that location. A gateway device (e.g., 150) may beimplemented as a dedicated gateway element, or may be a multi-purpose orgeneral purpose device, such as another IoT device (similar to devices105 a-d) or user device (e.g., 125, 130, 135) that itself may includesensors and/or actuators to perform tasks within an IoT system, amongother examples.

In some cases, applications can be programmed, or otherwise built orconfigured, utilizing interfaces of an IoT management system or adedicated IoT application development platform. In some cases, IoTapplication development tools can adopt asset abstraction to simplifythe IoT application building process. For instance, users can simplyselect classes, or taxonomies, of devices and logically assemble acollection of select devices classes to build at least a portion of anIoT application (e.g., without having to provide details regardingconfiguration, device identification, data transfer, etc.). IoTapplication development tools may further utilize asset abstraction todevelop and define deployment of one or more graphical user interfaces(GUIs) for use in a deployment of the resulting IoT application, toallow user control and management of a deployment during runtime.Further, IoT application systems built using the IoT management systemcan be shareable, in that a user can send data identifying theconstructed system to another user, allowing the other user to simplyport the abstracted system definition to the other user's environment(even when the combination of device models is different from that ofthe original user's system). Additionally, system or applicationsettings, defined by a given user, can be configured to be shareablewith other users or portable between different environments, among otherexample features.

In some cases, IoT systems can interface (through a corresponding IoTmanagement system or application or one or more of the participating IoTdevices) with remote services, such as data storage, informationservices (e.g., media services, weather services), geolocation services,and computational services (e.g., data analytics, search, diagnostics,etc.) hosted in cloud-based and other remote systems (e.g., 140, 145).For instance, the IoT system can connect to a remote service (e.g., 145)over one or more networks 120. In some cases, the remote service can,itself, be considered an asset of an IoT application. Data received by aremotely-hosted service can be consumed by the governing IoT applicationand/or one or more of the component IoT devices to cause one or moreresults or actions to be performed, among other examples.

One or more networks (e.g., 120) can facilitate communication betweensensor devices (e.g., 105 a-d), end user devices (e.g., 123, 130, 135),gateways (e.g., 150), and other systems (e.g., 140, 145) utilized toimplement and manage IoT applications in an environment. Such networkscan include wired and/or wireless local networks, public networks, widearea networks, broadband cellular networks, the Internet, and the like.

In general, “servers,” “clients,” “computing devices,” “networkelements,” “hosts,” “system-type system entities,” “user devices,”“gateways,” “IoT devices,” “sensor devices,” and “systems” (e.g., 105a-d, 125, 130, 135, 140, 145, 150, etc.) in example computingenvironment 100, can include electronic computing devices operable toreceive, transmit, process, store, or manage data and informationassociated with the computing environment 100. As used in this document,the term “computer,” “processor,” “processor device,” or “processingdevice” is intended to encompass any suitable processing apparatus. Forexample, elements shown as single devices within the computingenvironment 100 may be implemented using a plurality of computingdevices and processors, such as server pools including multiple servercomputers. Further, any, all, or some of the computing devices may beadapted to execute any operating system, including Linux, UNIX,Microsoft Windows, Apple OS, Apple iOS, Google Android, Windows Server,etc., as well as virtual machines adapted to virtualize execution of aparticular operating system, including customized and proprietaryoperating systems.

While FIG. 1A is described as containing or being associated with aplurality of elements, not all elements illustrated within computingenvironment 100 of FIG. 1A may be utilized in each alternativeimplementation of the present disclosure. Additionally, one or more ofthe elements described in connection with the examples of FIG. 1A may belocated external to computing environment 100, while in other instances,certain elements may be included within or as a portion of one or moreof the other described elements, as well as other elements not describedin the illustrated implementation. Further, certain elements illustratedin FIG. 1A may be combined with other components, as well as used foralternative or additional purposes in addition to those purposesdescribed herein.

As noted above, a collection of devices, or endpoints, may participatein Internet-of-things (IoT) networking, which may utilize wireless localarea networks (WLAN), such as those standardized under IEEE 802.11family of standards, home-area networks such as those standardized underthe Zigbee Alliance, personal-area networks such as those standardizedby the Bluetooth Special Interest Group, cellular data networks, such asthose standardized by the Third-Generation Partnership Project (3GPP),and other types of networks, having wireless, or wired, connectivity.For example, an endpoint device may also achieve connectivity to asecure domain through a bus interface, such as a universal serial bus(USB)-type connection, a High-Definition Multimedia Interface (HDMI), orthe like.

As shown in the simplified block diagram 101 of FIG. 1B, in someinstances, a cloud computing network, or cloud, in communication with amesh network of IoT devices (e.g., 105 a-d), which may be termed a“fog,” may be operating at the edge of the cloud. To simplify thediagram, not every IoT device 105 is labeled.

The fog 170 may be considered to be a massively interconnected networkwherein a number of IoT devices 105 are in communications with eachother, for example, by radio links 165. This may be performed using theopen interconnect consortium (OIC) standard specification 1.0 releasedby the Open Connectivity Foundation™ (OCF) on Dec. 23, 2015. Thisstandard allows devices to discover each other and establishcommunications for interconnects. Other interconnection protocols mayalso be used, including, for example, the optimized link state routing(OLSR) Protocol, or the better approach to mobile ad-hoc networking(B.A.T.M.A.N.), among others.

Three types of IoT devices 105 are shown in this example, gateways 150,data aggregators 175, and sensors 180, although any combinations of IoTdevices 105 and functionality may be used. The gateways 150 may be edgedevices that provide communications between the cloud 160 and the fog170, and may also function as charging and locating devices for thesensors 180. The data aggregators 175 may provide charging for sensors180 and may also locate the sensors 180. The locations, charging alerts,battery alerts, and other data, or both may be passed along to the cloud160 through the gateways 150. As described herein, the sensors 180 mayprovide power, location services, or both to other devices or items.

Communications from any IoT device 105 may be passed along the mostconvenient path between any of the IoT devices 105 to reach the gateways150. In these networks, the number of interconnections providesubstantial redundancy, allowing communications to be maintained, evenwith the loss of a number of IoT devices 105.

The fog 170 of these IoT devices 105 devices may be presented to devicesin the cloud 160, such as a server 145, as a single device located atthe edge of the cloud 160, e.g., a fog 170 device. In this example, thealerts coming from the fog 170 device may be sent without beingidentified as coming from a specific IoT device 105 within the fog 170.For example, an alert may indicate that a sensor 180 needs to bereturned for charging and the location of the sensor 180, withoutidentifying any specific data aggregator 175 that sent the alert.

In some examples, the IoT devices 105 may be configured using animperative programming style, e.g., with each IoT device 105 having aspecific function. However, the IoT devices 105 forming the fog 170device may be configured in a declarative programming style, allowingthe IoT devices 105 to reconfigure their operations and determine neededresources in response to conditions, queries, and device failures. Forexample, a query from a user located at a server 145 about the locationof a sensor 180 may result in the fog 170 device selecting the IoTdevices 105, such as particular data aggregators 175, needed to answerthe query. If the sensors 180 are providing power to a device, sensorsassociated with the sensor 180, such as power demand, temperature, andthe like, may be used in concert with sensors on the device, or otherdevices, to answer a query. In this example, IoT devices 105 in the fog170 may select the sensors on particular sensor 180 based on the query,such as adding data from power sensors or temperature sensors. Further,if some of the IoT devices 105 are not operational, for example, if adata aggregator 175 has failed, other IoT devices 105 in the fog 170device may provide substitute, allowing locations to be determined.

Further, the fog 170 may divide itself into smaller units based on therelative physical locations of the sensors 180 and data aggregators 175.In this example, the communications for a sensor 180 that has beeninstantiated in one portion of the fog 170 may be passed along to IoTdevices 105 along the path of movement of the sensor 180. Further, ifthe sensor 180 is moved from one location to another location that is ina different region of the fog 170, different data aggregators 175 may beidentified as charging stations for the sensor 180.

As an example, if a sensor 180 is used to power a portable device in achemical plant, such as a personal hydrocarbon detector, the device willbe moved from an initial location, such as a stockroom or control room,to locations in the chemical plant, which may be a few hundred feet toseveral thousands of feet from the initial location. If the entirefacility is included in a single fog 170 charging structure, as thedevice moves, data may be exchanged between data aggregators 175 thatincludes the alert and location functions for the sensor 180, e.g., theinstantiation information for the sensor 180. Thus, if a battery alertfor the sensor 180 indicates that it needs to be charged, the fog 170may indicate a closest data aggregator 175 that has a fully chargedsensor 180 ready for exchange with the sensor 180 in the portabledevice.

With the growth of IoT devices and system, there are increasing numbersof smart and connected devices available in the market, such as devicescapable of being utilized in home automation, factory automation, smartagriculture, and other IoT applications and systems. For instance, inhome automation systems, automation of a home is typically increased asmore IoT devices are added for use in sensing and controlling additionalaspects of the home. However, as the number and variety of devicesincrease, the management of “things” (or devices for inclusion in IoTsystems) becomes outstandingly complex and challenging.

Some device management applications may be configured with presetsettings designed to allow users to set up their systems quickly andwith minimal work configuring the device. However, those settings maynot be universally desirable, potentially limiting their flexibility andoverall utility. Indeed, the combination of settings that is ideal oroptimal to one user may be subjective, however, other users may still beinterested in the setting values determined by others which yieldedthese other users desirable results. In cases, where settings areconfigurable, a user may nonetheless struggle to make sense of the idealcombination of setting values, resorting to trial and error to figureout the optimal settings for their system (e.g., smart home, office,car, etc.). For instance, a guest of a first user may observe, firsthand, the configuration of the first user's system and desire torecreate the attributes of the observed system in their own system.However, in traditional systems, system settings are typically notportable due given that they are usually tied to identities of devices.Additionally, changing the parameters of an application's deployment mayrequire an existing service to be withdrawn and revised before a moredesirable deployment may be launched.

In traditional systems, each IoT device has its own device's uniquemedia access control (MAC) address or other device identifier.User-customized settings may be tied to a specified device present inthe system when the settings were first configured. However, intraditional systems, when a user replaces this device with another(e.g., newer) substitute device, the user is often required toreconfigure the device settings (including device identifier and deviceMAC address) in the application.

Given the dramatic growth in smart devices and promise of IoT systems,there have been considerable efforts in developing administrative toolsfor managing devices, data, resources, services, etc. in IoTapplication. Part of the promise of IoT systems, is that users canconfigure a customized IoT application specific to a user's home, anoffice space, or other environment. Tools have been developed such asIFTTT, SmartThings, Wink, and other systems have been developed, whichprovide a platform through which an end user may develop a new IoTservice using various sets of devices supported by each respectiveplatform. Such service development tools have been used to facilitatethe customized creation of services to realize automation applicationsuch as smart lighting, smart door lock, automated security, smartagriculture, etc.

While IoT development tools may assist in realizing new automationapplications bringing great convenience, comfort, energy efficiency andsecurity, one of the drawbacks of such systems (and the developmenttools used to program such applications) is that human involvement istypically limited purely to pre-deployment time, or development time.After developing a new IoT service using a traditional IoT developmenttool, however, the user-programmer is typically excluded from thedecision making loop within the service in post-deployment time, orruntime. While conventional end-to-end turnkey solutions continue to beinstalled and configured by experienced technicians or engineers, growthin IoT may be expected to be focused primarily on “home-brewed” IoTservices developed and installed by lay users with little or no formaltraining or experience. Novice users, while short on formal training,may be well-positioned to drive the design of IoT services given theirtypically superior knowledge of the domain in which the IoT service isto be deployed.

While some development tools have been tailored to simplifyingpre-deployment time for IoT users, limited access to runtime controlforces many users to iterate through the pre-deployment process in aninspect-revise cycle to get the post-deployment product they want it.For example, a user may use an IoT development tool to program a smartlighting script to use illuminance measures from a light sensor todetermine whether or not to turn a bulb on (e.g., the bulb will beturned on if the light sensor reading is less than 12,345; otherwise,the bulb will be turned off). While launching an IoT application basedon this script may be straightforward, troubleshooting and debugging thescript in post-deployment time may be more difficult. For instance,conventional IoT management tools may provide a pre-configured mechanismfor a user to inspect sensor values (e.g., through a fixed pre-definedGUI provided with a corresponding smartphone app), but lack the abilityfor a user to tune the presentation (visualization) to the particularcharacteristics of the IoT service and/or the preferences of theuser-developer. Further, while an IoT development may provide a userwith control to change and adjust an IoT service in pre-deployment time,traditional systems do not allow for humans to reconfigure parametersincorporated in the service logic (e.g., the light sensor readingthreshold 12,345) in the post-deployment time. Instead, traditional IoTdevelopment tools require a user to first “undeploy” the service,transition back to pre-deployment time and revise the program beforere-starting the corrected service.

In some implementations, an improved system may be provided withenhancements to address at least some of the example issues above. Forinstance, an improved IoT development platform may be provided thatenables the development and creation of IoT services based on definedasset abstractions. Asset abstraction may significantly reduce the humantouch points during deployment and redeployment and facilitate thedeployment of impromptu IoT systems, including IoT systems withcorresponding user interfaces allowing post-deployment involvement of auser. For instance, user interface abstraction may be incorporated in anasset abstraction scheme to allow automated and dynamic discovery anduse of devices an IoT application deployment. User interface abstractioncam provide a mechanism to address changes in the post-deployment time(without redeploying by going back and forth between pre-deployment andpost-deployment) and allow service developers to definewhat-you-see-is-what-you-get (WYSIWYG) abstracted user interfacesappropriate for the IoT service and enabling on-the-fly changes duringruntime.

With asset abstraction, IoT management and applications can adopt aparadigm where, instead of referencing and being programmed tointeroperate with specific IoT devices, the system can refer toabstracted classes, or taxonomies, of IoT devices (or “assets”). Assetabstraction can be leveraged to automatically configure a deployed IoTsystem (including corresponding UIs) with minimal human intervention.Indeed, in some instances, configuration of the system can progresswithout a user having to actually specify which device to use. Instead,a deployment policy can be used instead by the system to automaticallyselect and configure at least a portion of the devices within thesystem. Further, asset abstraction can facilitate addressing thechallenge of portability of IoT applications, which has traditionallylimited the general scalability and resiliency of IoT applications.

Asset abstraction can be coupled with automated asset binding, in somecases, to eliminate the necessity of including a device/asset's uniqueID in an IoT application or management program. Asset discovery providedwith the application or management program can provide an effectivemeans for specifying policy and confining the scope of asset binding.Through the combination of asset discovery, asset abstraction, and assetbinding makes IoT applications portable, reusable and shareable.

In some implementations, with asset abstraction, assets are treatedindifferently as long they fall into a same category in the taxonomy,e.g., occupancy sensing, image capture, computation, etc. An IoTapplication, consequently, can be made portable, reusable and shareable,as it can be written and stored in a way that specifies onlyrequirements (e.g., references to abstracted device taxonomies providingthe requirements) without specifying the precise identity (or catalogueof identities) of compatible devices meant to provide theserequirements. Asset discovery allows all available resources to besearched to detect those meeting the requirements and further selected,in some instances, on the basis of customizable or policy-basedcriteria.

Systems, such as those shown and illustrated herein, can include machinelogic implemented in hardware and/or software to implement the solutionsintroduced herein and address at least some of the example issues above(among others). For instance, FIG. 2 shows a simplified block diagram200 illustrating a system including multiple IoT devices (e.g., 105 a-b)with assets (e.g., sensors (e.g., 110 a,c) and/or actuators (e.g., 115a)) capable of being used in a variety of different IoT applications. Inthe example of FIG. 2 , a management system 205 is provided withdeployment manager logic 215 (implemented in hardware and/or software)to detect assets within a location and identify opportunities to deployan IoT system utilizing the detected assets. In some implementations, atleast a portion of the service logic (e.g., 202) utilized to drive thefunction of the IoT application may be hosted on the management system205. Service logic (e.g., 202) may also be hosted (additionally oralternatively) on one or more remote computing devices remote from themanagement system 205, including on devices (e.g., 105 a-b) utilizedwithin the IoT application deployment, servers within the environment inwhich the IoT application will be deployed, and/or cloud-based systems(remote from the location where the IoT application will be deployed).Configuration data (e.g., 204) to be used to configure the assets to beutilized in the deployment of the IoT system may also be hosted on themanagement system 205 or accessed by the management system 205 from oneor more other systems, including a user device (e.g., 130) or a remote(e.g., cloud-based) server, among other example implementations.

In the particular example of FIG. 2 , the management system 205 mayinclude one or more data processing apparatus (or “processors”) 208, oneor more memory elements 212, and one or more communication modules 214incorporating hardware and logic to allow the management system 205 tocommunicate over one or more networks (e.g., 120), utilizing one or moretechnologies (e.g., WiFi, Bluetooth, Near Field Communications, Zigbee,Ethernet, etc.), with other systems and devices (e.g., 105 a, 105 b,130, 210, etc.). The deployment manager 215 may be implemented utilizingcode executable by the processor 208 to manage the automated deploymentof a local IoT system. In some cases, the management system 205 may beimplemented on a dedicated physical system (e.g., separate from otherdevices in the IoT deployment). For instance, the management system 205may be implemented on a gateway device used to facilitate communicationwith and between multiple potential IoT devices (e.g., 105 a,b) within aparticular location. In some instances, the management system 205 may behosted at least in part on a user device (e.g., 130), including a userdevice that it utilized itself in the deployment of a particular IoTapplication. Indeed, portions of the management system 205 (anddeployment manager 215) may be implemented on multiple devices,including some devices remote from the environment in which the IoTapplication is to be deployed.

In one example, a deployment manager 215 may be provided that includeslogical components such as an asset discovery module 220, assetabstraction manager 225, asset binding manager 230 configuration manager235, runtime manager 240, and security engine 245, among other examplecomponents (or combinations of the foregoing). The deployment manager215 may access local (or remotely stored) service logic 202,configuration data 204, and user interface (UI) models 206, among otherexamples.

In one example, an asset discovery module 220 may be providefunctionality to allow the management system 205 to determine which IoTdevices (e.g., 105 a,b, 130) are within range of the management system205 and thus fall within a particular location for which one or more IoTservices may be deployed. In some implementations, the asset discoverymodule 220 makes use of the wireless communication capabilities (e.g.,214) of the management system 205 to attempt to communicate with deviceswithin a particular radius. For instance, devices within range of a WiFior Bluetooth signal emitted from the antenna(e) of the communicationsmodule(s) 214 of the management system 205 (or the communicationsmodule(s) (e.g., 288, 290) of the assets (e.g., 105 a,b)) can bedetected. Additional attributes can be considered by the asset discoverymodule 220 when determining whether a device is suitable for inclusionin a collection of devices for a given system or application. In someimplementations, conditions can be defined for determining whether adevice should be included in the collection. For instance, the assetdiscovery module 220 may attempt to identify, not only that it iscapable of contacting a particular asset, but may also determine assetssuch as physical location, semantic location, temporal correlation,movement of the device (e.g., is it moving in the same direction and/orrate as the discovery module's host), permissions or access levelrequirements of the device, among other characteristics. As an example,in order to deploy smart lighting control for every room in a home- oroffice-like environment, an application may be deployed in a “per roombasis.” Accordingly, the asset discovery module 220 can determine alisting of devices that are identified (e.g., through a geofence orsemantic location data reported by the device) as within a particularroom (despite the asset discovery module 220 being able to communicatewith and detect other devices falling outside the desired semanticlocation).

Conditions for discovery can be defined in service logic (e.g., 202) ofa particular IoT application. For instance, criteria can be defined toidentify which types of resources are needed or desired to implement anapplication. Such conditions can go beyond proximity, and includeidentification of the particular types of assets that the application isto use. For instance, the asset discovery module 220 may additionallyidentify attributes of the device, such as its model or type, throughinitial communications with a device, and thereby determine what assetsand asset types (e.g., specific types of sensors, actuators, memory andcomputing resources, etc.) are hosted by the device. Accordingly,discovery conditions and criteria can be defined based on assetabstractions (or asset taxonomies) defined for the IoT application. Somecriteria may be defined that is specific to a particular asset type,where the criteria has importance for some asset types but not forothers in the context of the corresponding IoT application. Further,some discovery criteria may be configurable such that a user cancustom-define at least some of the criteria or preferences used toselect which devices to utilize in furtherance of an IoT application.

A deployment manager 215 can also include an asset abstraction module225. An asset abstraction module 225 can recognize defined mappingsbetween specific IoT devices or, more generally, specific functionalitythat may be included in any one of a variety of present or future IoTdevices with a collection of defined taxonomies, or device abstractions.The asset abstraction module 225 can determine, for each assetdiscovered by an asset discovery module 220 (e.g., according to one ormore conditions), a respective asset abstraction, or taxonomy, to whichthe asset “belongs”. Each taxonomy can correspond to a functionalcapability of an asset. Assets known or determined to possess thecapability can be grouped within the corresponding taxonomy. Somemulti-function assets may be determined to belong to multiple of thetaxonomies. The asset abstraction module 225 can, in some cases,determine the abstraction(s) to be applied to a given asset based oninformation received from the asset (e.g., during discovery by assetdiscovery module 220). In some cases, the asset abstraction module canobtain identifiers from each asset and query a backend database forpre-determined abstraction assignments corresponding to that make andmodel of asset, among other examples. Further, in some implementations,the asset abstraction module 225 can query each asset (e.g., accordingto a defined protocol) to determine a listing of the capabilities of theasset, from which the asset abstraction module 225 can map the asset toone or more defined abstraction taxonomies. Asset abstraction module 225allows the application to treat every asset falling within a giventaxonomy as simply an instance of that taxonomy, rather than forcing thedeployment manager 215 to track every possible device model with whichit might be asked to manage or service logic 202 to be designed toconsider every possible permutation of a particular type of device.Asset abstraction module 225 can access a taxonomy framework (defined onan application-, system-, or universal-basis) that abstracts away theprecise device into taxonomies including higher- and lower-leveltaxonomies for sensing, actuating, computation, storage, and othertaxonomies. With asset abstraction, assets are treated indifferently aslong they fall into a same category in the taxonomy, e.g., occupancysensing. Deployment of an IoT application, implemented through itscorresponding service logic 202 and configurations 204, may be automatedin part through asset abstraction, allowing applications to be developedand deployed without concern for the specific identities of the devicesto be used in the system.

A deployment manager 205 can also include an asset binding module 230which can select, from the discovered assets, which assets to deploy fora system. In some cases, upon selecting an asset, the asset bindingmodule 230 can operate with configuration manager 235 to sendconfiguration information (e.g., 204) to selected assets to cause eachcorresponding asset to be configured for use in a particular service.This can involve provisioning the asset with corresponding service logiccode (e.g., to allow it to communicate and interoperate with thegateway, a service hosted by a backend server, and/or other assetsselected for deployment), logging in, unlocking, or otherwise enablingthe asset, sending session data to or requesting a session with theasset, among other examples. In cases where multiple assets of the sametaxonomy have been identified (and exceed a maximum desired number ofinstances of the taxonomy), the asset binding module 230 canadditionally assess which of the assets is the best fit for thedeployment. For instance, service logic (e.g., 202) may define bindingcriteria indicating desirable attributes of assets to be deployed in anapplication. These criteria can be global criteria, applying toinstances of every taxonomy, or can be taxonomy-specific (i.e., onlyapplying to decisions between assets within the same taxonomy). Assetbinding can provision the assets specified by the service logic (e.g.,202) for deployment automatically (before or during runtime).

A deployment manager 215 can additionally provide functionality (e.g.,through configuration manager 235) to allow configurations, or settings,to be applied to the selected asset taxonomies (or requirements) of theIoT application and the IoT application generally. A variety ofdifferent settings can be provided depending on the collection of assetsto be used by the application and the overall objectives of theapplication. Default setting values can be defined and further tools canbe provided to allow users to define their own values for the settings(e.g., a preferred temperature setting of an air conditioner, the numberof seconds to lock a smart lock or locker, sensitivity setting utilizedfor triggering a motion sensor and control, etc.). What settingsconstitute the “ideal” may be subjective and involve some tinkering bythe user. When the user is satisfied with the settings, the user maysave the settings as a configuration. In some implementations, theseconfigurations (e.g., 204) can be stored locally at the managementsystem 205, a device (e.g., 105 a,b), on an IoT gateway 150, or on thecloud. In some cases, configurations can be shared, such that a user canshare the settings they found ideal with other users (e.g., friends orsocial network contacts, etc.).

Configuration data can be generated from which the settings areautomatically readopted at runtime by the deployment manager 215, eachtime a corresponding service is to deploy (e.g., using whatever assetsare currently discoverable within a particular location). Consequently,while specific devices may only be loosely tied to any one user orgateway in a particular deployment of a service, settings can bestrongly tied to a user or service, such that the user may migratebetween environments and the service may be deployed in variousenvironments, including environments with different sets of assets, withthe same settings, or configuration, being applied in each environment.For instance, regardless of the specific device identifiers orimplementations selected to satisfy the abstracted asset requirements ofan application or service, the same settings can be applied (e.g., asthe settings, too, are directed to the abstractions of the assets (i.e.,rather than specific assets)). To the extent a particular setting doesnot apply to a selected instance of a taxonomy, the setting can beignored. If a selected instance of a taxonomy possesses settings thatare undefined by the user in the configuration (e.g., because they areunique to the particular asset), default values for these settings canbe automatically set or the user can be alerted that these settings areundefined, among other examples.

A configuration manager 235 may be additionally used in runtime (e.g.,during and following deployment of an IoT system) to cause particularsettings to be applied at the IoT devices (assets) selected fordeployment with the service. The deployment manager 215 may includelogic enabling the deployment manager 215 (and its composite modules) tocommunicate using a variety of different protocols with a variety ofdifferent devices. Indeed, the deployment manager 215 can even be usedto translate between protocols to facilitate asset-to-assetcommunications. Further, the configuration manager 235 can sendinstructions to each of the selected assets for deployment to prompteach asset to adjust settings in accordance with those defined for theasset taxonomy in the setting configuration defined in configurationdata pushed to (or pulled from) the configuration manager 235 during(and potentially also after) deployment.

A system utilizing a gateway enhanced with deployment manager 215 may beenabled to combine automatic resource management/provisioning withauto-deployment of services. A configuration manager 235 can allowresource configurations from one IoT system to be carried over andapplied to another so that services can be deployed in various IoTsystems. Further, a runtime manager 240 can be utilized to performautomated deployment and management of a service resulting from thedeployment at runtime. Auto-configuration can refer to the configurationof devices with configurations (e.g., 204) stored locally or on a remotenode, to provide assets (and their host devices) with the configurationinformation to allow the asset to be properly configured to operatewithin a corresponding IoT application deployment. As an example, adevice may be provided with configuration information usable by thedevice to tune a microphone sensor asset on the device so that is mightproperly detect certain sounds for use in a particular IoT system (e.g.,tune the microphone to detect specific voice pitches with improvedgain). Auto-deployment of a services may involves identification (ordiscovery) of available devices, device selection (or binding) based onservice requirements (configuration options, platform, and hardware),and automated continuous deployment (or re-deployment) to allow theservice to adapt to evolving conditions.

In one example, a runtime manager 240 may be utilized to direct thedeployment and running of a service on a set of devices within aparticular location (e.g., a location corresponding to a gatewayimplementing the management system 205). In one example, runtime manager240 may trigger asset discovery and binding (e.g., by asset discoverymodule 220 and asset binding manager 230) in connection with thedeployment of a particular application. Runtime manager 240 may accessservice logic 202 for a particular application, or service, and maycommunicate with deployed devices (e.g., 105 a-b, 130, etc.) to senddata to the devices (e.g., to prompt certain actuators) or receive data(e.g., sensor data) from the devices in accordance with thecorresponding service logic 202. A runtime manager 240 may also utilizeservice logic 202 and provide received data as inputs to the logic anduse the service logic 202 to generate results, including results whichmay be used to prompt certain actuators on the deployed devices. In somecases, the generation of these results may include utilizing remotelyexecuted service logic, for instance, by sending a request to a backendservice Runtime manager logic 240 may also be utilized in connectionwith security management logic 245, to define security domains within adeployment, for instance, to secure communications between one or moreof the deployed devices and a gateway and/or communications between thedevices themselves.

Portions of the application, or service logic, can be distributed duringdeployment, with service logic capable of being executed locally at thegateway (or even one of the deployment computing assets) and/or remotefrom the deployment location on a cloud-based or other remotely-locatedsystem. Indeed, in some cases, the management system 205 (e.g., usingruntime manager 240) may provide one or more assets or their hostdevices (e.g., 105 a,b) with service logic for use during an IoTapplication's deployment. In some cases, the management system 205 (andruntime manager 240) may manage deployment and execution of multipledifferent applications (e.g., with corresponding sets of service logic202). Different configurations (e.g., using different configuration datainstances) of the same application may also be supported by a singlemanagement system (or gateway). Once assets are provisioned, thedeployed assets can be used collectively for achieving the goals andfunctionality designed for the application.

In some IoT applications, one or more UI assets may be defined, anddevices may be deployed within a deployment of the IoT application toprovide one or more UIs to facilitate runtime management of thedeployment. In some cases, UI logic and functionality may already bepresent on a UI asset (e.g., of a user device or other user interfaceplatform (e.g., a smart home control monitor), etc.), while in othercases UI logic may be provisioned on a selected asset on-demand by themanagement system to supplement or to be used by UI logic andfunctionality already on the device. For instance, UI models (e.g., 206)may define various types of UIs and UI features which may be provisionedon UI assets in connection with an IoT application deployment. IoTapplication UIs may include graphical user interfaces (GUIs) throughwhich a user receives outputs and/or provides inputs (e.g., through atouchscreen), voice interfaces (through which voice commands can bereceived), gesture-based interfaces (e.g., utilizing a camera to captureuser gestures), audio interfaces (e.g., through which data is presentedvia speakers on a device), among other examples. A UI model can providelogic to assist in the implementation of any one of such UIs.

As noted above, user devices (e.g., 130) may be utilized in a variety ofways within an IoT application deployment. User devices may possessmanagement system functionality, functionality of an IoT servicedevelopment system (e.g., 210), may be utilized to control or manage aparticular IoT application (e.g., through a UI of the IoT applicationprovided on the device 130), or to provide other assets (e.g., sensor,actuator, computing, or storage) for use in a particular IoT applicationdeployment. In one example, a user device 130 may include a UI engine265, which may be leveraged in a particular IoT application deploymentto provide one or more UIs for use by a user in connection with thedeployment. A user device 130 may include one or more data processors(e.g., 266), one or more memory elements (e.g., 268), a communicationmodule 270 enabling communication with other systems using wirelessand/or wireline network connections, and an operating system 272 onwhich one or more applications (e.g., 274) may be run. A user device 130may include one or more input devices 276, which may embody sensorsimplementing a touchscreen interface, keyboard, tracker ball, camera, orother mechanism through which user inputs may be captured. A user device130 may also include one or more presentation devices 278 (e.g., drivenby corresponding actuators) to implement a graphical display, an audiopresentation (e.g., speakers), a light output (e.g., of an integratedLED flashlight or camera flash), or vibration motor to outputvibration-based signals, among other examples. Input devices andpresentation devices, together with UI engine 265 driving operation ofthese devices) and storage (e.g., 268) and computing (e.g., 266)resources of a user device (e.g., 130) may be utilized to fulfill UIrequirements of a particular IoT application, resulting in thedeployment of a user device (e.g., 130) in connection with deployment ofthe particular IoT application.

As noted above, asset abstraction can assist not only in easing thedeployment of a system and propagating configurations across multipledifferent systems, but abstraction may also be used to enhance theprogramming of IoT applications. For instance, a development system 210may be provided, through which a user may define and create a new IoTapplication, or service. The development system 210 may be provided withtraditional textual and/or graphical programming tools (e.g., for use incoding the application) and enhanced with declarative programming toolsallowing users, including novice programmers, to program a service byspecifying generalized or abstracted requirements of the IoTapplication, expressed as collections of asset taxonomies. The user mayadditionally declare relationships between two or more of the selectedtaxonomies to indicate input-output, sensor-actuator, or otherrelationships that are to be automatically defined in correspondingapplication code and thereby develop new IoT applications (andcorresponding service logic (e.g., 202) based on these abstractions.Such a development paradigm may allow an IoT application to beprogrammed once and deployed theoretically anywhere (in varyingenvironments using varying combinations of devices).

In one example, a development system 210 can include one or moreprocessors 246 and one or more memory elements 248 utilized to implementa development engine 250. In some cases, the development system 210 canbe integrated with a user device (e.g., 130) and/or management system(e.g., 205). The development engine 250 can provide an interface throughwhich a user may generate service logic 202 corresponding to an IoTapplication to be deployed in one or more environments. The developmentengine 250 may include a requirement definition 255 component to allowusers to define the service, at least in part, by defining a set ofasset requirements and relationships between the asset requirements.Asset requirements may be defined according to asset abstractions (e.g.,built from a library of asset abstraction definitions 260) and allow auser to define the service according to capability abstractions, whichin turn, during deployment, may be translated into resourceabstractions, which may map to real devices discovered (e.g., by thedeployment manager 215) during a particular deployment of the service.Some of the asset abstraction definitions 260 may include UI assetabstractions to further allow users to define UI requirements for aservice created using the development engine 250. The resulting servicelogic (e.g., 202) may likewise allow UI platforms to be deployedtogether with the remaining IoT application deployment in an ad hoc andimpromptu manner (e.g., based on the present set of available devicesand assets discoverable by a management system (e.g., 205) in connectionwith a given deployment. Indeed, as a general manner, IoT services maybe developed using asset abstraction definitions through developmentsystem 210 in advance of and independent of the identities of thephysical devices that will be used to implement the service, and thephysical deployments may be later and automatically deployed based onthe defined abstractions in the corresponding service logic.

Continuing with the description of FIG. 2 , each of the IoT devices(e.g., 105 a,b) may include one or more processors (e.g., 284, 285), oneor more memory elements (e.g., 286, 287), and one or more communicationsmodules (e.g., 288, 290) to facilitate their participation in variousIoT application deployments. Each device (e.g., 105 a,b) can possessunique hardware, sensors (e.g., 110 a,c), actuators (e.g., 115 a), andother logic (e.g., 292, 294) to realize the intended function(s) of thedevice (including operation of the respective sensors and actuators).For instance, devices may be provided with such resources as sensors ofvarying types (e.g., 110 a, 110 c), actuators (e.g., 115 a) of varyingtypes, energy modules (e.g., batteries, solar cells, etc.), computingresources (e.g., through a respective processor and/or software logic),security features, data storage, and other resources.

Turning to FIG. 3A, a simplified block diagram 300 a is shownrepresenting a simplified example of asset abstraction. A variety ofdifferent taxonomies can be defined at varying levels. For instance, asensor taxonomy can be a parent to a multitude of specific sensor-typetaxonomies (e.g., child taxonomies for light sensing, motion sensing,temperature sensing, liquid sensing, noise sensing, etc.), among otherexamples. In the example of FIG. 3A, an IoT application has been definedto include three asset requirements, represented by taxonomies MotionSensing 305 a, Computation 305 b, and Alarm 305 c. During assetdiscovery, a variety of assets (e.g., 308 a-f) can be identified asusable by the application (e.g., based on the assets meeting one or moredefined discovery conditions). One or more corresponding taxonomies, orabstractions, can be identified (e.g., by an IoT management system) foreach of the assets 308 a-f. Some of the abstractions may not haverelevance to the asset requirements and function of the application,such as an abstraction (e.g., Temperature Sensor and/or HVAC Actuator)determined for thermostat device 308 f. Other asset abstractions maymatch the abstractions (e.g., 305 a-c) designated in the IoT applicationas asset requirements of the application. Indeed, more than onediscovered asset may be fit one of the asset requirements. For instance,in the example of FIG. 3A, a PIR sensor 308 a and camera 308 b are eachidentified as instances of a motion sensing asset taxonomy 305 a.Similarly, a cloud-based computing resource 308 c and network gateway308 d are identified as instances of a computation asset taxonomy 305 b.In other instances, there may be just a single discovered devicesatisfying an application asset requirement (e.g., siren 308 e of thealarm taxonomy 305 c), among other examples.

Conventionally, IoT and wireless sensor network (WSN) applications havebeen developed to intricately define dataflow among a determined set ofphysical devices, which involves device-level discovery in developmenttime to obtain and hardcode the corresponding device identifiers andcharacteristics. By utilizing asset abstraction, development can befacilitated to allow the devices to be discovered and determined atruntime (e.g., at launch of the application), additionally allowing theapplication to be portable between systems and taxonomy instances.Further, development can be expedited by allowing developers to merelyspecify asset requirements (e.g., 305 a-c), without the necessity tounderstand radio protocol, network topology, and other technicalfeatures.

In one example, taxonomies for asset abstraction can involve such parenttaxonomies as sensing assets (e.g., light, presence, temperaturesensors, etc.), actuation (e.g., light, HVAC, machine controllers,etc.), power (e.g., battery-powered, landline-powered, solar-powered,etc.), storage (e.g., SD, SSD, cloud storage, etc.), computation (e.g.,microcontroller (MCU), central processing unit (CPU), graphicalprocessing (GPU), cloud, etc.), and communication (e.g., Bluetooth,ZigBee, WiFi, Ethernet, etc.), among other potential examples.Discovering which devices possess which capabilities (and belong towhich taxonomies) can be performed using varied approaches. Forinstance, some functions (e.g., sensing, actuating, communication) maybe obtained directly from signals received from the device by the systemmanagement system via a common descriptive language (e.g., ZigBee'sprofiles, Bluetooth's profiles and Open Interconnect Consortium'sspecifications), while other features (e.g., power, storage,computation) may be obtained through deeper queries (utilizing resourceson top of the operating system of the queried device or through queriesto a remote service), among other examples.

Asset binding can be applied to determine which discovered assets(fitting the asset requirements (abstractions) defined for anapplication) are to actually be deployed. Criteria can be defined atdevelopment time and/or before/at runtime by the application's user,which an IoT deployment manager (e.g., 205) can consult to perform thebinding. For instance, as shown in FIG. 3A, according to the criteriaset forth for the application (or for a particular session using theapplication), one of multiple matching assets for a required taxonomycan be selected. For instance, between PIR sensor 308 a and camera 308b, corresponding criteria (e.g., criteria to be applied generally acrossall taxonomies of the application and/or taxonomies specific to themotion sensing taxonomy 305 a) can result in PIR sensor 308 a beselected to be deployed to satisfy the motion sensing asset requirement305 a of the application. Similarly, criteria can be assessed todetermine that gateway 308 d is the better candidate between it andcloud resource 308 c to satisfy the application's computationrequirement 305 b. For asset requirements (e.g., 305 c) where only asingle discovered instance (e.g., 308 e) of the asset taxonomy isdiscovered, asset binding is straightforward. Those discovered devices(e.g., 308 a, 308 d, 308 e) that have been selected, or bound, can thenbe automatically provisioned with resources from or configured by theIoT deployment manager (e.g., 205) to deploy the application. Unselectedassets (e.g., 308 b, 308 c, 308 f) may remain in the environment, butare unused in the application. In some instances, unselected assets canbe identified as alternate asset selections (e.g., in the event of afailure of one of the selected assets), allowing for swift replacementof the asset (deployed with the same settings designated for instancesof the corresponding taxonomy).

In some instances, asset binding can be modeled as a bipartite matching(or assignment) problem in which the bipartite graph can be expressed byG=(R,A,E) where R denotes the asset requirements, A denotes theavailable assets and e=(r,a) in E where a in A is capable of r in R.Note that if R requests for n instances of a particular assets, A′ canbe defined as:

$\bigcup\limits_{n}A$from which a solution for the (maximum) (weighted) matching problem canbe computed. For instance, exhaustive search can be applied as thenumber of vertices in the bipartite graph are small and the edges areconstrained in the sense that there is an edge (r,a) only if a iscapable of r.

Turning to the simplified block diagram 300 b of FIG. 3B, an example ofasset discovery is represented. Asset discovery can allow the scope ofavailable devices to be confined based on discovery conditions orcriteria, such as conditions relating to device proximity, room,building, movement state, movement direction, security, permissions,among many other potential (and configurable) conditions. The benefitsof such targeted discovery can trickle down to asset binding, asunchecked discovery may return many possible bindings, especially inlarge scale deployment. For example, in a smart factory, the action of“deploying predictive maintenance” may be ambiguous as there may behundreds of sensors, motors, alarms, etc. in a factory facility. Assetdiscovery, in some implementations, takes as input a policy or userinput from which a set of discovery criteria can be identified. Upondetecting the universe of assets with which the application couldpotentially operate, the criteria can be used to constrain the set, insome cases, providing a resulting ordered list of available assets,which can be expressed as f:C×D

D, where C denotes criteria, D denotes a set of devices, and thecodomain is a totally ordered set.

For instance, in the example of FIG. 3B, two discovery criteria 315 a,315 b are identified for an application. Additional criteria may bedefined that is only to apply to some or a specific one of thecategories, or taxonomies, of assets, among other examples. Based on thedefined criteria 315 a-b in this example, the output of discoveryaccording to search criteria A 315 a leads to the codomain of a subsetof devices in the environment—LS1 (310 a), LS2 (310 b), GW2 (310 g) andLA1 (310 h), whereas search criteria B results in LS2 (310 b), LS3 (310c), TS1 (310 d), HS1 (310 e), GW1 (310 f), and LA1 (310 h). Based on theset of defined discovery criteria (e.g., 315 a-b), asset discovery canattempt to reduce the total collection of identified assets to a bestsolution. Additionally, determining the set of discovered assets forbinding consideration can incorporate determining a minimum set ofdiscovered devices, based on the asset requirements of the application.For instance, a minimum set can be selected during discovery such thatat least one asset of each required taxonomy is present in the set, ifpossible. For instance, in the example of FIG. 3B, it can be identified(e.g., by an asset discovery module of the deployment manager) thatapplication of only criteria B (315 b) in discovery yields at least oneasset for each of the taxonomies defined for the application.

For instance, the block diagram 300 c FIG. 3C illustrates the end-to-enddeployment determinations of a deployment manager for a particular IoTapplication 350. For instance, based on the discovery conducted in theexample of FIG. 3B, a subset of the assets (e.g., LS2 (310 b), LS3 (310c), TS1 (310 d), HS1 (310 e), GW1 (310 f), and LA1 (310 h)) are“discovered” for potential use by the application (e.g., based on theircompliance with criteria B (and the underrepresentation of assets incompliance with criteria)). Accordingly, assets LS1 and GW2 are not tobound to the corresponding IoT application 350 (as indicated by thedashed lines (e.g., 330)), despite each asset being an instance of oneof the asset requirements (e.g., Light Sensing, Compute, and Storage) ofthe application 350.

As noted above, additional criteria can be defined and applied duringasset binding. During binding, where the set of discovered assetsinclude more than one instance of a particular required asset taxonomy(e.g., as with assets L2 and L3 in asset taxonomy Light Sensing),criteria can be applied to automatically select the asset that is thebetter fit for deployment within the IoT system governed, controlled, orotherwise supported by the application 350. Further, as illustrated inFIG. 3C, it is possible for a single asset instance (e.g., GW1) to bothbelong to two or more taxonomies and to be selected for binding to theapplication for two or more corresponding asset requirements (e.g.,Compute and Storage), as shown. Indeed, a binding criterion can bedefined to favor opportunities where multiple asset requirements of theapplication can be facilitated through a single asset, among otherexamples.

As represented generally in FIG. 3C, asset discovery can provide thefirst level for confining the scope of an asset-to-application assetrequirement mapping. A user or developer can specify (in some cases,immediately prior to runtime) the asset requirements for a particularapplication 350, and an environment can be assessed to determine whetherassets are available to satisfy these asset requirements. Further, thedeployment manager utility can automatically deploy and provisiondiscovered assets to implement that application, should the requisitecombination of assets be found in the environment. Additionally, thedeployment manager utility can automatically apply setting values acrossthe deployed assets in accordance with a configuration defined by a userassociated with the application. However, if no instances of one or moreof the asset requirements (required taxonomies) are discovered, theapplication may be determined to be un-deployable within theenvironment. In such cases, a deployment manager utility can generate analert for a user to identify the shortage of requested taxonomyinstances, including identifying those taxonomies for which no assetinstance was discovered within the environment, among other examples.

Turning to FIGS. 4A-4B, simplified block diagrams 400 a-b are shownrepresenting example IoT abstraction layers used in one implementationof an IoT application (pre- and post-deployment). For instance, in theexample of FIG. 4A, the abstractions may include UI abstractions (e.g.,405, 410, 415), as well as logic execution abstractions (e.g., 420),sensor abstractions (e.g., 425), and actuator abstractions (e.g., 430),among other examples. Various abstraction layers may be definedincluding a capability abstraction layer 435, a resource abstractionlayer 440, and runtime abstraction layer 445. The capability abstractionlayer 435 may be utilized during programming of a service, with eachcapability abstraction corresponding to a capability or functionalitythat is to be incorporated in an IoT service. Each capabilityabstraction may further map to one or more resource abstractions in aresource abstraction layer 440. Each resource abstraction may correspondto a type or category of asset (abstracting the precise device model,MAC, serial number, etc.), such as a light sensor, IP camera, display,computing resource, storage resource, power switch, microphone,temperature sensor, smart thermostat, etc.

During deployment of the service, a management system may determine amapping between capability abstractions and the resource abstractionswhich map to the capability abstraction. In other cases, the servicelogic generated from the capability abstractions selected duringdevelopment may include or derive the mapping and execution of theservice logic may cause the resource abstractions to be identified tothe management system, among other example implementations. Themanagement system may utilize resource abstractions during assetdiscovery to determine which assets potentially satisfy the requirements(e.g., specified by the capability abstractions) of the correspondingIoT application. As noted above, asset binding may involve themanagement system selecting one of potentially multiple satisfactoryassets for a particular resource abstraction. For instance, in theexample of FIG. 4A, a Light Sensing capability 425 may map to twopotential resource abstractions, Light Sensors and IP Cameras. Two lightsensors, L1 and L2, may be discovered in an environment (by a managementsystem) together with two distinct IP cameras (e.g., IC1, IC2). Themanagement system, in this particular example, selects light sensor L1for the deployment (e.g., based on one or more binding criteria), andthe selected light sensor L1 is mapped to the runtime abstraction LightSense 450. Upon selecting (and binding) specific, discovered devices,the service logic may interact with the selected asset by reference toits respective runtime abstraction (in runtime abstraction layer 445).Each runtime abstraction may correspond to one of the capabilityabstractions defined for the IoT application, and may represent theruntime, post-deployment view of the IoT service vis-à-vis the deployedassets.

IoT solutions typically define IoT resources as atomic resources, inthat each capability maps to a single type of device. However, somecapabilities may be between defined through a recognition that a singlecapability is, in fact, realized through multiple different resources incombination (or a “compound” of resources). Other capabilities mayremain realized by a single atomic resource. In the context of variousUI capabilities, a UI capability may be defined as a compound ofresources. For example, as shown in the partial view of the mapping 400a shown in FIG. 4B, the capability abstraction 405, “UI Log,” may map tothree resource abstractions (i.e., a compound resource), namely Display,Computing, and Storage. In other words, a one-to-many relationship mayexist between a capability abstraction and corresponding resourceabstractions. For atomic resources, such as for a Light Sensingcapability abstraction 425, a one-to-one capability-to-resource mappingmay be made. When binding discrete physical devices, or assets, toresource abstractions, a single device (such as smartphone SP2) may mapto multiple resource abstractions. Indeed, a single device may satisfy acompound resource, such as shown in the example of FIG. 4B.

In the example of FIG. 4A, three UI capability abstractions are includedin an example IoT service definition. For instance, a UI Dial capabilityabstraction may be provided that identifies a desire for the IoTapplication deployment to include a UI input realized through adial-like UI element (e.g., a WYSIWYG dial for adjusting a particularvalue used in the IoT service). In this example, to implement the UIDial capability, a touchscreen resource and a computing resource may beutilized and, in some cases, within an all-in-one device, which can beexpressed as follows:C_(UI) _(dial) ≅{r_(computing),r_(display)}∈C_(UI) _(IN)where C_(UI) _(dial) denotes the capability of UI Dial, C_(UI) _(IN)denotes UI Input (the collection of all UIs for input) andr_(computing),r_(display) represent the resources of computing anddisplay, respectively. Other UI input capabilities may be definedaccording to a similar basis, with each distinct UI input capabilityrepresenting a particular style or type of input GUI element (e.g.,dial, switch, button, slider, among potentially limitless othervarieties). Output UI capabilities may be likewise diverse in varietyand may be generally defined similar to output UI capability UI Log:C_(UI) _(log) ≅{r_(computing),r_(storage),r_(display)}∈C_(UI) _(OUT)where C_(UI) _(log) denotes the capability of UI Log, C_(UI) _(OUT)denotes UI Output (the collection of all UIs for output) andr_(computing),r_(storage),r_(display) represent the resources ofcomputing, storage and display, respectively. Other output UIcapabilities (different from a log-style UI output) may also be definedand included in a library of capability abstractions from which an IoTservice may be defined and developed. Further, while the examples inFIG. 4A-4B show display-centric UI capability abstractions, in someimplementations, UIs may include non-display UIs, such as UIs thatutilize voice/speech recognition technology, audio output, gesturerecognition, tactile or vibration outputs, among other UI examples.

As noted above, asset abstractions in the capability abstraction layer430 may be utilized during pre-deployment time, or development time, inthe creation of an IoT service. For instance, as illustrated in theexample screenshot 500 a of FIG. 5A, an IoT service development tool maysupport graphical development of an IoT service. For instance, a usermay select (e.g., from a library of capability abstractions) variousgraphical blocks (e.g., 505, 510, 515, 520, 525, 530) corresponding tovarious capability abstractions (and capabilities), which the userdesires to include within the IoT system, and drag and drop these blocksinto a development window 535. The user may additionally draw orotherwise define connectors between the blocks to define relationshipsand interactions between the various selected blocks (e.g., 505, 510,515, 520, 525, 530). For instance, in the particular example of FIG. 5A,a smart lighting control service is being developed (with capabilitiessimilar to those shown in the example illustrated in FIGS. 4A-4B), witha light sensor capability 505 being linked to a logical executioncapability 510, which performs a comparison between a value measured bythe light sensor capability 505 with a threshold value defined by a UIinput capability 515 (set using a dial UI element). In conventionalsystems, the threshold value may be determined and set as a constantduring programming of the IoT service. In the example of FIG. 5A,however, a UI is provided for deployments of the IoT service to allowthis threshold value to be modified post-deployment by a user. The logicexecution capability 510 may then provide a result of its comparison toa light switch capability 520. Additional UI capabilities 525, 530 maybe provided to facilitate UI outputs and provide a log that recordslight sensor values (e.g., at 525) and results (e.g., true/false,on/off, etc.) of logic execution capability 510, among other examples.

Logic execution capabilities 510, like UI capabilities, can take many(potentially limitless) forms specific to the corresponding IoT servicebeing defined. Indeed, a logic execution capability (e.g., 510) mayembody some of the service logic that it utilized to define a function,calculation, or processing step that is to be performed on datagenerated by one or more sensor capabilities and potentially provided toactuator assets to realize one or more actions or activities of the IoTservice (e.g., automating the turning on of a light). For instance, alogic execution capability may provide pre- or post-processing of sensordata and/or inputs received through UI assets, including processing ofmultiple values returned in data generated by one or more other assetsin the IoT service.

Using a graphical programming (or a textual programming) paradigm, auser may define the capabilities of a new IoT service together withrelationships between the capabilities using capability abstractions, asshown in the example of FIG. 5A. Service logic may be generated by theprogramming tool to embody the capabilities. This service logic may thenbe used in any number of different, automated deployments of thecorresponding IoT service (using potentially many different combinationsof devices satisfying the capability abstraction requirements of the IoTservice). For example, as illustrated in the block diagram 500 b of FIG.5B, one example deployment of a service corresponding to the exampleservice developed based on the graphical programming represented in FIG.5A, may result in the binding of a particular light sensor device 550 toa light sensing capability (e.g., 505) defined in the service and anautomated light switch 555 may be bound to a light switch capability(e.g., 520) defined in the service. A first smartphone device 560 may beselected as including the display, computing, and storage resourcesmapped to the “UI Dial” UI capability and “Less Than” logic executioncapability abstractions (e.g., as shown in FIG. 4A), while a secondsmartphone device 565 is selected as providing the display, computing,and storage resources mapped to the “UI Log” capability abstractionsdefined for the IoT service. In other cases, a single smartphone may beselected to replace the two separate smartphones selected in thisparticular example, so long as the single smartphone includes theresources satisfying the resource abstraction requirements mapped to theIoT service's respective capability abstractions (e.g., UI Dial, UI Log,“Less Than” Logic, etc.).

Turning to FIG. 6 , as noted above, an application developed accordingto the principles of asset abstraction, as described herein, can allow agiven IoT application to be deployed in a number of locations employingvaried collections of IoT devices and assets. Such IoT applications, orservices, may be deployed spontaneously in response to detecting thatsufficient assets exist within range of a management system (e.g., on aparticular gateway device (e.g., 150 a,b). Upon identifying andselecting resources for the deployment, the management system mayprovide configurations to the selected assets for use in definingcharacteristics of the assets during a particular deployment of the IoTapplication. In some cases, different configurations can be employed indifferent deployments of the same IoT applications, leading potentially,to different outcomes in each deployment (including in deployments thatare otherwise identical (e.g., using the same combination of IoT devicesin a comparable environment)). In other cases, the same configurationscan be employed in distinct deployments that utilize differentcombinations of devices (e.g., different devices bound to at least someof the defined abstractions of the IoT application) to yield comparableoutcomes, even when the devices used are not identical. As an example, auser can define preferred settings in a particular configuration for aparticular IoT application and cause different deployments of theparticular IoT application (in other locations) to also adopt the user'spreferred settings when they are deployed. For instance, frequenttravelers may prefer to create a home-like environment (e.g., in ahotel, office, or vacation rental remote from their home) by causingparticular configuration data defined by the user (e.g., hosted on acloud 605) to be applied to each environment in which deployable devicesexist. Accordingly, the particular configuration data (e.g., 204) can beused to deploy multiple instances of an IoT application (withpotentially different devices) all with the same settings.

For instance, as shown in the simplified block diagram 600 of FIG. 6 ,in a first environment 610, a first gateway 150 a can be utilized todeploy a first instance of a particular IoT application. Service logic(e.g., 204) to implement the IoT application may be hosted on gateway150 a and/or remotely by an application server or other system(s)providing cloud resources (e.g., 605). In one example, a smartphone 130(or other device) may enter the first environment 610 and communicatewith a corresponding gateway 150 a to indicate that a first set ofconfigurations should be employed in a deployment of an IoT applicationin the first environment 610. For instance, the gateway 150 a may deploythe IoT application in the first environment 610 by discovering a set ofassets A1, A2, B1, C1, D1, D2, where assets A1 and A2 are instances oftaxonomy A, asset B1 is an instance of taxonomy B, and so on. Furtherthe same (or a different) smartphone may also be detected as providingresources to satisfy one or more UI capabilities defined for the IoTapplication. In this example, another smartphone 135 is selected tosatisfy the resources required for one or more UI (input and/or output)capabilities defined for the IoT application, while smartphone 130 isnot selected for inclusion in the deployment in environment A 610. Forthe remaining capability abstractions of The IoT application, such ascapability abstractions A, B, C, D, and E, additionally asset selection(or binding) may be performed, resulting in assets A1, B1, C1, D1 and E1being selected and deployed for the instance of the IoT applicationdeployment in Environment A 610. Additionally, a particular set ofconfigurations (defined in configuration data 208) may be pushed to theselected assets (A1, B1, C1, D1, E1, and smartphone 135) for use in thedeployment. In some cases, a UI model, script, or other data may bepushed to the resources (e.g., 135) selected to provide the UIcapabilities for the deployment.

In some examples, the use of a particular set of configurations may bebased on a request of a user or even the identification (e.g., by thegateway) that a particular user device associated with a user is presentin Environment A 405. Accordingly, the gateway can configure an IoTapplication's particular deployment based on the preferences of a userwithin the environment, a property owner (e.g., a manager or owner ofthe environment), according to government or corporate regulations,among other examples. By remotely hosting configuration data, theseconfigurations may be portable in the sense that multiple differentmanagement systems (e.g., 150 a and 150 b) in multiple, different (andpotentially remote) environments may attempt to apply the sameconfigurations to different deployments of the same particular IoTapplication. In other cases, the configurations pushed to the respectiveassets in each deployment of the particular IoT application (e.g., inEnvironment A (610) and Environment B (615)) may be different.

Continuing with the example of FIG. 6 , In another, remote environment,Environment B (615), an instance of the same IoT application may bedeployed by another gateway 150 b in another environment 615. Adifferent set of assets may be discovered in Environment B 615 than wasused in Environment A 610, resulting in a different set of deployedassets (e.g., A2, B2, C2, D1, and E1) for the IoT application inEnvironment B 615. Some of the assets in Environment B may be instancesof the same asset (e.g., the same device model) discovered inEnvironment A (e.g., A2, C1, D1). Some assets may not be strongly tiedto location, such as assets on a mobile device (e.g., 130, 135) or othermobile device that may be used in both the IoT application deploymentsin Environments A and B. Despite the deployments being different betweenthe two environments (e.g., 610, 615), when viewed at the assetabstraction level, the deployments may be functional equivalents.Further, the settings utilized in each deployment can be applied equallywithin each environment 605, 610 by providing the same configurations(from configuration data 204) to each of the respective IoT applicationdeployments. As an example, a sensitivity setting value defined in theconfiguration data for the taxonomy instance B can be applied equally toeach of assets B1 and B2 (and, for that matter, B3, when deployed),allowing the deployment managers of the gateways to attempt to achieveequivalent systems utilizing different collections of assets. While, inpractice, the resulting systems may not be functionally identical (asdifferences between the asset instances (e.g., B1 and B2) may manifest,even when configured with the same settings), implementations of theapplication in varied environments can be at least approximated withminimal effort of the user.

Further, in the example of FIG. 6 , a smartphone device 130 that was notselected for use as a UI resource in a first deployment of an IoTapplication (e.g., the deployment in Environment A (610)), the samesmartphone device 130 (e.g., when moved to another environment anddiscovered by a corresponding management system (e.g., gateway 150 b))may be bound to UI resources (and mapped-to capability abstractions) inother deployments. For instance, the smartphone 130 may be identified,in Environment B by the gateway 150 b, as the only device having theresources that satisfied one or more of the UI capability abstractionsfor the IoT application or better satisfied binding criteria establishedfor the IoT application or the particular deployment of the applicationin Environment B (e.g., as defined in configuration data 208 or servicelogic 204), among other examples.

Service logic developed using an abstraction-based programming tool mayadditionally allow new devices to be discovered and included in anapplication deployment during runtime of the deployment. For instance, amanagement system may detect that a new device enters a network and maydetermine the assets available on the new device. The management system,upon detecting that at least some of the assets correspond to resourceabstractions mapped to the capability abstractions defined for theapplication, may communicate with the new device to configure the device(and its assets) for addition to the deployment. Indeed, for UI assets,a new or additional UI may be provided, based on detection of a newdevice supporting the resources (e.g., display, storage, and computing)defined for a corresponding UI capability abstraction. For instance, inthe example of FIG. 6 , device 135 may be initially identified andselected for use in satisfying one or more UI capability abstractionrequirements of an application in Environment A 610. However, use device130 may later enter Environment A 610 and be added dynamically as anadditional or substitute UI asset. In some instances, where multiple UIcapability abstractions are defined for an application, one device(e.g., 135) may be selected to map to the resource requirements of afirst one of the UI capability abstractions, while a second one of thedevices (e.g., 130) is selected to fulfill the remaining UI capabilityabstractions. These mappings of devices to capability (and runtime)abstractions may change too, as device enter or exit (or fail) duringruntime, with other devices being selected to supplement or replace thedeployed devices in response.

Turning to FIG. 7 , a simplified block diagram 700 is shown illustratingan example deployment of an IoT application using a deployment manager(e.g., 215) implemented on a user device, IoT gateway, controllerdevice, or other system. The deployment manager 215, in one example, mayinclude an IoT server 705 to serve one or more IoT applications usingvarious combinations of devices (e.g., 105 a, 105 b, 130) discoverablewithin a particular location or network. The deployment manager 215 mayadditionally include a local configuration/service manager 710, whichmay manage access to local and remote configuration data and servicelogic, as well as the provisioning of devices (e.g., 105 a, 105 b, 130)in an IoT deployment with the same. Local configuration data and servicelogic may be stored in local storage 415 of the gateway 150. Additionalconfiguration data and service logic may also be accessed and utilizedby the deployment manager 215 by accessing (e.g., throughconfiguration/service manager 710) one or more remote configurationand/or service managers (e.g., collectively 720) serving configurationdata and/or service logic (e.g., at data store(s) 725).

Selection of the particular configuration data or service logic (e.g.,from remote or local configuration storage) to be used in the deploymentmay be controlled by a user through user device or other controller,among other examples. Whether the configuration data or service logicoriginates from remote storage (e.g., 725), from a user device (e.g.,130), a device (e.g., 105 a) to be potentially included in the deployedIoT application, storage 715 local to the deployment manager 215, oranother source, the configuration data or service logic is to be firstprovided to the configuration/service manager 710 on the deploymentmanager 215 before being used for runtime deployment.

The deployment manager 215 itself may be configured to deploy variousinstances of the same IoT applications or of different IoT applications.Settings of each of these deployment instances may be pre-programmed ormay be provided by an administrator (e.g., through an application on auser device (e.g., 130) in communication with the deployment manager215), and these settings may drive what service logic and configurationdata the gateway utilizes in runtime deployment of an IoT application.For instance, service manager (e.g., 710) of the deployment manager 215may determine that particular service logic is to be employed for aparticular IoT application deployment. The service logic may be accessedfrom local storage 715 and may, in some cases, indicate coordinationwith one or more other systems (e.g., 720) running additional servicelogic remote from the deployment manager 215, which the local servicemanager 710 is to request and utilize in certain circumstances andtransactions. Upon identifying the service logic to use, the servicemanager 710 can provide the service logic 730 to the runtime manager 240to initiate deployment of a corresponding IoT application.

Service logic (and the services provided through its execution) maydefine interactions between devices and the actions that are to beperformed through the IoT application's deployment. The service logicmay identify the assets required or desired within the deployment andmay identify the same by asset abstraction (e.g., by capabilityabstraction). For instance, as implied in the example of FIG. 7 , one ormore UI capabilities may be defined for IoT service deployments, and oneor more user devices (e.g., 130) may be discovered and selected tosatisfy these abstracted requirements of the IoT service. Further,interactions or relationships between the devices may also be defined,with these definitions, too, being made by reference to respective assetabstractions. Accordingly, the service logic can define the set ofdevices (or device types) that is to be discovered by the gateway anddrive the discovery and binding processes used to identify and select aset of devices (e.g., 105 a, 105 b, 130) to be used in the deployment.

The service logic may be carried out locally by the runtime manager 250.In some cases, the service can be implemented as a script and beutilized to trigger events and actions utilizing the deployed devices.As noted above, the service logic may also identify conditions whereoutside computing resources, such as a system hosting remote servicelogic is to be called upon, for instance, to assist in processing datareturned by sensors in IoT application deployment. Services (performedlocally or remotely through corresponding service logic) may include thereceiving of inputs, such as sensor readings, static values or actuatortriggers, functions, processes, and calculations to be applied to theinputs, UI inputs, and outputs generated based on the function results,which may in turn specify certain actions to be performed by an actuatoror results to be presented on a user device (e.g., 130), among otherexamples. In some cases, portions of service logic may be distributed tocomputing resources, or assets, within the deployed IoT application, anda portion of the input processing and result generation for a deploymentmay be performed by computation assets on the deployed devices (e.g.,105 a, 105 b, 130) themselves. Such results may in turn be routedthrough or returned to the gateway for further processing (e.g., byservice logic local to the deployment manager 215 or by service logicexecuted on a cloud-based backend system (e.g., 720), etc.).

Service deployment may begin with running the provided service logic onthe runtime manager 250. The runtime manager 250 can process the logicto identify a set of asset abstractions mapped to requirements of theIoT application. Identification of these abstractions may promptinitiation of an asset discovery stage 740 (e.g., performed through theIoT Server 705). During discovery, devices (e.g., 105 a, 105 b, 130,etc.) may be discovered together with identifier information of eachdevice to allow the deployment manager 215 to determine which assetabstraction(s) may be mapped to each device (with its respectivecollection of assets). In one example, the runtime manager 250 may firstdetermine a set of resource abstractions mapped to the set of capabilityabstractions defined for a particular application and then determinewhich of the discovered assets satisfy which of the determined resourceabstractions for the particular application). In the event that moreassets of a particular type are identified within the location than areneeded, the gateway can additional perform a binding analysis (accordingto one or more binding criteria) to select which device(s) to bind toone or more corresponding asset abstractions.

With the set of devices selected for a corresponding IoT applicationdeployment, automated configuration of the devices may be performed bythe deployment manager 215. Configuration data may embody aconfiguration that identifies one or more static settings relevant to aparticular device to which the configuration is being applied. Multipleconfigurations may be provided for use in provisioning multipledifferent types of devices in a given deployment. Various configurationdata in data stores may describe multiple, different presetconfigurations, each tailored to a specific scenario or deployment.Configuration data may be provided in a standard format, such as XML,JSON or CBOR file, among other examples. The local configuration manager(e.g., 710) may handle the deployment of configurations on thediscovered devices (e.g., 105 a, 105 b, 130). The communication betweenthe deployment manager 215 and the devices (e.g., 105 a, 105 b, 130) maybe handled by the IoT server component 705. Configuration data is sentfrom the local configuration manager to corresponding devices discoveredand selected for deployment in a particular IoT application, Dependingon the number of devices, hardware capabilities of the local gateway 150and the local network bandwidth and latency each device (e.g., 105 a,105 b, 130) in a deployment, the deployment manager 215 may either sendconfiguration data directly to each of the IoT devices in the deploymentor may utilize one or more of the devices to distribute configurationdata to other devices in a peer-to-peer fashion, among other examples.

The runtime manager 250 may define messages (e.g., 735) to be passed tothe various devices in range of the deployment manager 215 to facilitatediscovery of the devices (as well as ongoing discovery of new devicesfollowing deployment) and delivery of configuration data to theappropriate devices (e.g., 105 a, 105 b, 130). In some cases, theconfiguration manager 710 of the deployment manager 215 can identify theappropriate configurations corresponding to the discovered devices(e.g., 105 a, 105 b, 130) and provide corresponding configuration data(at 745) to the IoT server 705 for distribution (directly or through P2Psharing) of the configuration data to the appropriate devices.

For devices (e.g., 105 a) including sensor assets (e.g., 110 a),configuration data may indicate to the asset (and its host device) howit is to collect and relay data using the sensors. For instance,configuration data may be utilized by the host device to cause data tobe collected, generated, or reported at a particular interval, to tunethe sensor to certain settings, to apply a filter to the informationsensed by the sensor, among other examples. For devices (e.g., 105 b)including actuator assets (e.g., 115 a), configuration data may indicateto the asset how to respond to inputs (e.g., provided through thegateway) and perform activities in response to these requests. Forinstance, the configuration data for an actuator asset may cause anactuator to wait for certain commands or messages 750, 755 (e.g., fromthe IoT server 705) before an action, to tune its actions to particularsettings, select which actions are performed or how they are performedthrough the actuator, among other examples. Configuration data may alsobe sent to assets tasked with satisfying UI capabilities of theapplication, with configuration data potentially driving the style ortype of UI elements provided to a user through a device (e.g., 130) inconnection with the deployment.

With the configuration data 745 provided to the discovered devices(e.g., 105 a, 105 b, 130) initial deployment may be considered completeand devices (e.g., 105 a, 105 b, 130) and their respective assets (e.g.,110 a, 115 a, 266, 268, 278, etc.) may operate in accordance with theconfigurations provided them. Accordingly, during runtime, sensingmessages (e.g., 750) may be sent up to the deployment manager 215 fromthe devices (e.g., 105 a, 105 b). Data from UI inputs and data to berendered in UI outputs (e.g., 755) may also be transmitted. In oneexample, runtime manager 250 can receive the sensing messages (at 735)and utilize service logic either local to or remote from the deploymentmanager 215 to process the sensor data as inputs. In someimplementations, the runtime manager 250 may redirect or forward sensordata to one or more deployed computing (e.g., 266) or storage (e.g.,268) assets for processing on another one of the deployed devices (e.g.,130), among other examples. One or more results may be generated fromthe processing and used as the basis of actuating messages sent from theruntime manager 250 to the IoT server (at 735) for delivery (at 750) toone or more specific actuator assets (e.g., 115 a on devices 105 b) tocause a corresponding activity to be performed. Results of theprocessing may also be provided to assets providing UI outputfunctionality (e.g., at 755). In this manner, a system of IoT servicedeployment and resource management may be provided which enablesauto-configuration of devices and fast deployment of services making useof those configurations, including services with UI features allowingruntime control of the deployment by one or more users, among otherexamples.

While some of the systems and solution described and illustrated hereinhave been described as containing or being associated with a pluralityof elements, not all elements explicitly illustrated or described may beutilized in each alternative implementation of the present disclosure.Additionally, one or more of the elements described herein may belocated external to a system, while in other instances, certain elementsmay be included within or as a portion of one or more of the otherdescribed elements, as well as other elements not described in theillustrated implementation. Further, certain elements may be combinedwith other components, as well as used for alternative or additionalpurposes in addition to those purposes described herein.

Further, it should be appreciated that the examples presented above arenon-limiting examples provided merely for purposes of illustratingcertain principles and features and not necessarily limiting orconstraining the potential embodiments of the concepts described herein.For instance, a variety of different embodiments can be realizedutilizing various combinations of the features and components describedherein, including combinations realized through the variousimplementations of components described herein. Other implementations,features, and details should be appreciated from the contents of thisSpecification.

FIGS. 8A-8B are simplified flowcharts 800 a-b illustrating exampletechniques for developing and deploying an IoT application or othermachine-to-machine system. For instance, in the example of FIG. 8A, adevelopment tool may be provided through which user inputs may bereceived 805. The user inputs may cause a set of capability abstractionsto be selected (e.g., from a library of capability abstractions) todefine 810 functionality to be included in a new IoT application. Thedevelopment tool may be further used to define 815 relationships betweenthe selected set of capability abstractions. Service logic may beautomatically generated 820 from the defined set of capabilityabstractions that is executable to perform deployment of the IoTapplication in any one of a plurality of different environments andpotentially using varying collections of specific devices and assets.

For instance, in the example of FIG. 8B, a gateway or other IoTmanagement system may be utilized to discover 830 a plurality of deviceswithin an environment. The IoT management system may determine 835assets present on the discovered devices and select devices for aparticular deployment of an IoT application based on the assets meetingone or more asset abstraction requirements (e.g., capabilityabstractions) defined for the a particular IoT application (e.g., theapplication developed in the example of FIG. 8A). In some cases, thegateway may make these determinations (e.g., 835) autonomously, while inother cases the gateway may consume services of a remote system inconnection with making the determination 835. Selection of the devicesmay include determining 840 a mapping between runtime abstractions(which map to capability abstractions of the IoT application) and theselected assets. At least some of these runtime abstractions cancorrespond to UI capability abstractions, and corresponding assets maybe identified by the IoT management system to implement UIs in an IoTapplication deployment that enable runtime manipulation of thedeployment by one or more users. The IoT management system may interactwith, access logic for, and/or identify the participating devices, notby their specific identities, but rather by their runtime abstractions.Runtime abstractions may represent the interface between the IoTmanagement system (and application logic) and the devices within aparticular deployment. Further, in some cases, multiple deployments ofthe same application (even within the same location or environment) maybe supported by an IoT management system and distinct deployments mayeach have their own set of runtime abstractions. Further, configurationdata may be identified for the deployment that is likewise mapped toruntime abstractions. The IoT management system may delivercorresponding portions of the configuration data to the selected assetsmapped to these runtime abstractions to deploy 845 the particularapplication in the environment using the selected assets. Followingdeployment, sensor data may be generated by deployed sensor assets andthis sensor data may be used by computation logic and/or actuator assetsdeployed in the IoT application and UI outputs may provide views intoactuator status, sensor data values, and the results of computationlogic, among other examples.

FIGS. 9-10 are block diagrams of exemplary computer architectures thatmay be used in accordance with embodiments disclosed herein. Othercomputer architecture designs known in the art for processors andcomputing systems may also be used. Generally, suitable computerarchitectures for embodiments disclosed herein can include, but are notlimited to, configurations illustrated in FIGS. 9-10 .

FIG. 9 is an example illustration of a processor according to anembodiment. Processor 900 is an example of a type of hardware devicethat can be used in connection with the implementations above. Processor900 may be any type of processor, such as a microprocessor, an embeddedprocessor, a digital signal processor (DSP), a network processor, amulti-core processor, a single core processor, or other device toexecute code. Although only one processor 900 is illustrated in FIG. 9 ,a processing element may alternatively include more than one ofprocessor 900 illustrated in FIG. 9 . Processor 900 may be asingle-threaded core or, for at least one embodiment, the processor 900may be multi-threaded in that it may include more than one hardwarethread context (or “logical processor”) per core.

FIG. 9 also illustrates a memory 902 coupled to processor 900 inaccordance with an embodiment. Memory 902 may be any of a wide varietyof memories (including various layers of memory hierarchy) as are knownor otherwise available to those of skill in the art. Such memoryelements can include, but are not limited to, random access memory(RAM), read only memory (ROM), logic blocks of a field programmable gatearray (FPGA), erasable programmable read only memory (EPROM), andelectrically erasable programmable ROM (EEPROM).

Processor 900 can execute any type of instructions associated withalgorithms, processes, or operations detailed herein. Generally,processor 900 can transform an element or an article (e.g., data) fromone state or thing to another state or thing.

Code 904, which may be one or more instructions to be executed byprocessor 900, may be stored in memory 902, or may be stored insoftware, hardware, firmware, or any suitable combination thereof, or inany other internal or external component, device, element, or objectwhere appropriate and based on particular needs. In one example,processor 900 can follow a program sequence of instructions indicated bycode 904. Each instruction enters a front-end logic 906 and is processedby one or more decoders 908. The decoder may generate, as its output, amicro operation such as a fixed width micro operation in a predefinedformat, or may generate other instructions, microinstructions, orcontrol signals that reflect the original code instruction. Front-endlogic 906 also includes register renaming logic 910 and scheduling logic912, which generally allocate resources and queue the operationcorresponding to the instruction for execution.

Processor 900 can also include execution logic 914 having a set ofexecution units 916 a, 916 b, 916 n, etc. Some embodiments may include anumber of execution units dedicated to specific functions or sets offunctions. Other embodiments may include only one execution unit or oneexecution unit that can perform a particular function. Execution logic914 performs the operations specified by code instructions.

After completion of execution of the operations specified by the codeinstructions, back-end logic 918 can retire the instructions of code904. In one embodiment, processor 900 allows out of order execution butrequires in order retirement of instructions. Retirement logic 920 maytake a variety of known forms (e.g., re-order buffers or the like). Inthis manner, processor 900 is transformed during execution of code 904,at least in terms of the output generated by the decoder, hardwareregisters and tables utilized by register renaming logic 910, and anyregisters (not shown) modified by execution logic 914.

Although not shown in FIG. 9 , a processing element may include otherelements on a chip with processor 900. For example, a processing elementmay include memory control logic along with processor 900. Theprocessing element may include I/O control logic and/or may include I/Ocontrol logic integrated with memory control logic. The processingelement may also include one or more caches. In some embodiments,non-volatile memory (such as flash memory or fuses) may also be includedon the chip with processor 900.

FIG. 10 illustrates a computing system 1000 that is arranged in apoint-to-point (PtP) configuration according to an embodiment. Inparticular, FIG. 10 shows a system where processors, memory, andinput/output devices are interconnected by a number of point-to-pointinterfaces. Generally, one or more of the computing systems describedherein may be configured in the same or similar manner as computingsystem 1000.

Processors 1070 and 1080 may also each include integrated memorycontroller logic (MC) 1072 and 1082 to communicate with memory elements1032 and 1034. In alternative embodiments, memory controller logic 1072and 1082 may be discrete logic separate from processors 1070 and 1080.Memory elements 1032 and/or 1034 may store various data to be used byprocessors 1070 and 1080 in achieving operations and functionalityoutlined herein.

Processors 1070 and 1080 may be any type of processor, such as thosediscussed in connection with other figures. Processors 1070 and 1080 mayexchange data via a point-to-point (PtP) interface 1050 usingpoint-to-point interface circuits 1078 and 1088, respectively.Processors 1070 and 1080 may each exchange data with a chipset 1090 viaindividual point-to-point interfaces 1052 and 1054 using point-to-pointinterface circuits 1076, 1086, 1094, and 1098. Chipset 1090 may alsoexchange data with a high-performance graphics circuit 1038 via ahigh-performance graphics interface 1039, using an interface circuit1092, which could be a PtP interface circuit. In alternativeembodiments, any or all of the PtP links illustrated in FIG. 10 could beimplemented as a multi-drop bus rather than a PtP link.

Chipset 1090 may be in communication with a bus 1020 via an interfacecircuit 1096. Bus 1020 may have one or more devices that communicateover it, such as a bus bridge 1018 and I/O devices 1016. Via a bus 1010,bus bridge 1018 may be in communication with other devices such as auser interface 1012 (such as a keyboard, mouse, touchscreen, or otherinput devices), communication devices 1026 (such as modems, networkinterface devices, or other types of communication devices that maycommunicate through a computer network 1060), audio I/O devices 1014,and/or a data storage device 1028. Data storage device 1028 may storecode 1030, which may be executed by processors 1070 and/or 1080. Inalternative embodiments, any portions of the bus architectures could beimplemented with one or more PtP links.

The computer system depicted in FIG. 10 is a schematic illustration ofan embodiment of a computing system that may be utilized to implementvarious embodiments discussed herein. It will be appreciated thatvarious components of the system depicted in FIG. 10 may be combined ina system-on-a-chip (SoC) architecture or in any other suitableconfiguration capable of achieving the functionality and features ofexamples and implementations provided herein.

Although this disclosure has been described in terms of certainimplementations and generally associated methods, alterations andpermutations of these implementations and methods will be apparent tothose skilled in the art. For example, the actions described herein canbe performed in a different order than as described and still achievethe desirable results. As one example, the processes depicted in theaccompanying figures do not necessarily require the particular ordershown, or sequential order, to achieve the desired results. In certainimplementations, multitasking and parallel processing may beadvantageous. Additionally, other user interface layouts andfunctionality can be supported. Other variations are within the scope ofthe following claims.

In general, one aspect of the subject matter described in thisspecification can be embodied in methods and executed instructions thatinclude or cause the actions of identifying a sample that includessoftware code, generating a control flow graph for each of a pluralityof functions included in the sample, and identifying, in each of thefunctions, features corresponding to instances of a set of control flowfragment types. The identified features can be used to generate afeature set for the sample from the identified features

These and other embodiments can each optionally include one or more ofthe following features. The features identified for each of thefunctions can be combined to generate a consolidated string for thesample and the feature set can be generated from the consolidatedstring. A string can be generated for each of the functions, each stringdescribing the respective features identified for the function.Combining the features can include identifying a call in a particularone of the plurality of functions to another one of the plurality offunctions and replacing a portion of the string of the particularfunction referencing the other function with contents of the string ofthe other function. Identifying the features can include abstractingeach of the strings of the functions such that only features of the setof control flow fragment types are described in the strings. The set ofcontrol flow fragment types can include memory accesses by the functionand function calls by the function. Identifying the features can includeidentifying instances of memory accesses by each of the functions andidentifying instances of function calls by each of the functions. Thefeature set can identify each of the features identified for each of thefunctions. The feature set can be an n-graph.

Further, these and other embodiments can each optionally include one ormore of the following features. The feature set can be provided for usein classifying the sample. For instance, classifying the sample caninclude clustering the sample with other samples based on correspondingfeatures of the samples. Classifying the sample can further includedetermining a set of features relevant to a cluster of samples.Classifying the sample can also include determining whether to classifythe sample as malware and/or determining whether the sample is likelyone of one or more families of malware. Identifying the features caninclude abstracting each of the control flow graphs such that onlyfeatures of the set of control flow fragment types are described in thecontrol flow graphs. A plurality of samples can be received, includingthe sample. In some cases, the plurality of samples can be received froma plurality of sources. The feature set can identify a subset offeatures identified in the control flow graphs of the functions of thesample. The subset of features can correspond to memory accesses andfunction calls in the sample code.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinventions or of what may be claimed, but rather as descriptions offeatures specific to particular embodiments of particular inventions.Certain features that are described in this specification in the contextof separate embodiments can also be implemented in combination in asingle embodiment. Conversely, various features that are described inthe context of a single embodiment can also be implemented in multipleembodiments separately or in any suitable subcombination. Moreover,although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the embodiments described above should not be understoodas requiring such separation in all embodiments, and it should beunderstood that the described program components and systems cangenerally be integrated together in a single software product orpackaged into multiple software products.

The following examples pertain to embodiments in accordance with thisSpecification. One or more embodiments may provide a system, a machinereadable storage medium, an apparatus, and/or method to define, based onone or more user inputs received through a graphical user interface of aprogramming tool, a set of capability abstractions for a particularapplication and further define, based on the one or more user inputs,relationships between the set of capability abstractions. The particularapplication is to utilize a machine-to-machine network, and the set ofcapability abstractions can include: at least one sensor capabilityabstraction, at least one actuator capability abstraction, at least onecomputation logic capability abstraction, at least one input userinterface (UI) capability abstraction, and at least one output UIcapability abstraction for the particular application. The relationshipscan include a relationship between the input UI capability abstractionand the computation logic capability abstraction, where the input UIcapability is to provide an input to the computation logic capabilityabstraction. Executable code including service logic may be generated todeploy an instance of the particular application in an environment,where the code is generated based on the set of capability abstractionsand the relationships.

In one example, deploying the instance of the particular applicationincludes selecting one or more devices in the environment for inclusionin the deployed instance of the particular application to implement auser interface for the particular application, and the user interface isprovided based on at least one of the input UI capability abstractionand the output UI capability abstraction.

In one example, the input UI capability abstraction includes a firstinput UI capability abstraction and the set of capability abstractionsfurther includes a second input UI capability abstraction.

In one example, the first input UI capability abstraction corresponds toproviding a first type of input UI element in the user interface and thesecond input UI capability abstraction corresponds to providing adifferent, second type of input UI element in the user interface.

In one example, the user interface enables runtime modifications to thedeployed instance of the particular application.

In one example, the service logic is executable to identify a set ofresource abstractions corresponding to the set of capabilityabstractions, and identify a collection of assets in the environmentsatisfying the set of resource abstractions.

In one example, the service logic is further to select at least aportion of the collection of assets for deployment in the instance ofthe particular application.

In one example, selection of the portion of the collection of assets isbased on respective proximity of the assets to a management systemcorresponding to the environment.

In one example, the management system includes a gateway to communicatewirelessly with the collection of assets.

In one example, the service logic is executed at least in part at thegateway.

In one example, at least a particular one of the set of capabilityabstractions maps to a plurality of resource abstractions in the set ofresource abstractions and another one of the set of capabilityabstractions maps to a single one of the set of resource abstractions.

In one example, the input UI capability abstraction and output UIcapability abstraction each map to a respective plurality of resourceabstractions in the set of resource abstractions.

In one example, the service logic identifies the collection of assetsfollowing deployment of the instance of the particular applicationaccording to respective runtime abstractions mapped to the set ofcapability abstractions.

In one example, the set of capability abstractions are selected from alibrary of capability abstractions and the library of capabilityabstractions includes a plurality of different input UI capabilityabstractions, and a plurality of different output UI capabilityabstractions.

In one example, the relationships include a relationship between theoutput UI capability abstraction and one of the sensor capabilityabstraction, actuator capability abstraction, and computation logiccapability abstraction.

In one example, the computation logic capability abstraction correspondsto performance of an operation on sensor data to be generated by assetscorresponding to the sensor capability abstraction.

In one example, the service logic is reusable to deploy the particularapplication in another environment using a different collection ofassets.

In one example, the one or more user inputs are received through agraphical user interface of an application programming tool.

One or more embodiments may provide a system, a machine readable storagemedium, an apparatus, and/or method to detect a plurality of deviceswithin an environment including a user computing device, identify a setof assets hosted on each of the plurality of devices, determine amapping between each of the assets in the subset of assets and arespective one of a set of runtime asset abstractions defined for aparticular application, and deploy the particular application using theplurality of devices. The set of runtime asset abstractions may includeat least one output user interface (UI) abstraction, at least one inputUI abstraction, at least one sensor asset abstraction, at least oneactuator asset abstraction, and at least one computation logic assetabstraction, and assets of the user computing device may be mapped tothe output UI abstraction, the input UI abstraction, and the computingasset abstraction.

In one example, the set of runtime asset abstractions correspond to aset of capability abstractions defined in code of the particularapplication.

In one example, the code further defines relationships between the setof capability abstractions.

In one example, the particular application is deployed using theplurality of devices in a first deployment and the particularapplication is deployed using a different plurality of devices accordingto the set of capability abstractions in a subsequent second deployment.

In one example, the instructions are further executable to identifyparticular configuration data corresponding to deployment of theparticular application, and deploying the particular applicationincludes sending the particular configuration data to the plurality ofdevices to configure assets of the plurality of devices for runtime ofthe particular application.

In one example, each of the input UI capability abstraction and outputUI capability abstraction map to a respective plurality of asset types.

In one example, a system may include a system manager and the systemmanager may be hosted at least in part on a gateway configured tocommunicate with the plurality of assets.

Thus, particular embodiments of the subject matter have been described.Other embodiments are within the scope of the following claims. In somecases, the actions recited in the claims can be performed in a differentorder and still achieve desirable results. In addition, the processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults.

The invention claimed is:
 1. A server comprising: at least one storagedevice including first instructions; and at least one processor toexecute second instructions to cause the first instructions to bedistributed via a network, the first instructions to cause at least onecomputing device to: cause presentation of a graphical user interface toenable a user to build a graphical representation of a flow for anInternet of Things (IoT) system including a first electronic device anda second electronic device; generate, based on one or more user inputs,a first block in the graphical representation of the flow, the firstblock associated with a first device type, the first device typeassociated with a first capability; associate, based on the one or moreuser inputs, the first electronic device to implement the first blockfor the first capability of the first device type in the graphical userinterface; generate, based on the one or more user inputs, a secondblock in the graphical representation of the flow, the second blockassociated with a second device type, the second device type associatedwith a second capability, the second block to be positioned relative tothe first block in the flow in response to a drag-and-drop input;associate, based on the one or more user inputs, the second electronicdevice to implement the second block for the second capability of thesecond device type in the graphical user interface; generate, based onthe one or more user inputs, a line between the first and second blocksin the graphical representation of the flow, the line indicative of arelationship or interaction between the first electronic deviceassociated with the first block and the second electronic devicesassociated with represented by the first and the second blocks;generate, based on the one or more user inputs, a third block in thegraphical representation of the flow, the third block associated with alog, wherein the flow for the IoT system is to cause state informationfor the second electronic device to be stored in the log; and causedeployment of the flow for the IoT system, the second capability of thesecond device type to enable a user to interact with the secondelectronic device to impact operation of the first electronic deviceafter deployment based on the relationship and or interaction definedbetween the first and second electronic devices, the interaction toactivate an operational state of the first electronic device based on astate of the second electronic device, the first capability based on amodel of the first electronic device, the first instructions to causethe at least one computing device to determine a first plurality ofelectronic devices based on the first capability, the first plurality ofelectronic devices available for selection for association to implementthe first block, the first plurality of electronic devices include thefirst electronic device, the first instructions to cause the at leastone computing device to determine a second plurality of electronicdevices based on the second capability, the second plurality ofelectronic devices available for selection for association to implementthe second block, the second plurality of electronic devices includingthe second electronic device.
 2. The server of claim 1, wherein thefirst capability is to enable a display of information from the IoTsystem via a screen of the first electronic device, the informationbased on the relationship or interaction between the first and secondelectronic devices.
 3. The server of claim 1, wherein the interaction isto activate the first electronic device based on a state of the secondelectronic device.
 4. The server of claim 1, wherein the interaction isto activate the first electronic device based on a comparison of a stateof the second electronic device to a state of a user input device. 5.The server of claim 1, wherein the first device type is an output andthe second device type is an input.
 6. The server of claim 1, whereinthe state information includes an indication of a current output of thefirst block.
 7. One or more computer readable storage devices comprisinginstructions that, when executed, cause at least one processor to atleast: cause presentation of a graphical user interface to enable a userto build a graphical representation of a flow for an Internet of Things(IoT) system including a first electronic device and a second electronicdevice; generate, based on one or more user inputs, a first block in thegraphical representation of the flow, the first block associated with afirst device type, the first device type associated with a firstcapability; associate, based on the one or more user inputs, the firstelectronic device to implement the first block for the first capabilityof the first device type in the graphical user interface; generate,based on the one or more user inputs, a second block in the graphicalrepresentation of the flow, the second block associated with a seconddevice type, the second device type associated with a second capability,the second block to be positioned relative to the first block in theflow in response to a drag-and-drop input; associate, based on the oneor more user inputs, the second electronic device to implement thesecond block for the second capability of the second device type in thegraphical user interface; generate, based on the one or more userinputs, a line between the first and second blocks in the graphicalrepresentation of the flow, the line indicative of a relationship orinteraction between the first electronic device associated with thefirst block and the second device type associated with the second block;generate, based on the one or more user inputs, a third block in thegraphical representation of the flow, the third block associated with alog, wherein the flow for the IoT system is to cause state informationfor the second electronic device to be stored in the log; and causedeployment of the flow for the IoT system, the second capability of thesecond device type to enable a user to interact with the secondelectronic device to impact operation of the first electronic deviceafter deployment based on the relationship or interaction between thefirst and second electronic devices, the interaction to activate thefirst electronic device based on a state of the second electronicdevice, the first capability based on a model of the first electronicdevice, the instructions to cause the at least one processor todetermine a first plurality of electronic devices based on the firstcapability, the first plurality of electronic devices available forselection for association to implement the first block, the firstplurality of electronic devices include the first electronic device, theinstructions to cause the at least one processor to determine a secondplurality of electronic devices based on the second capability, thesecond plurality of electronic devices available for selection forassociation to implement the second block, the second plurality ofelectronic devices including the second electronic device.
 8. The one ormore computer readable storage devices of claim 7, wherein the firstcapability is to enable a display of information from the IoT system viaa screen of the first electronic device, the information based on therelationship or interaction between the first and second electronicdevices.
 9. The one or more computer readable storage devices of claim7, wherein the first capability is based on a model of the firstelectronic device.
 10. The one or more computer readable storage devicesof claim 7, wherein the interaction is to activate the first electronicdevice based on a state of the second electronic device.
 11. The one ormore computer readable storage devices of claim 7, wherein theinteraction is to activate the first electronic device based on acomparison of a state of the second electronic device to a state of auser input device.
 12. The one or more computer readable storage devicesof claim 7, wherein the first device type is an output and the seconddevice type is an input.
 13. The one or more computer readable storagedevices of claim 7, wherein the state information includes an indicationof a current output of the first block.
 14. A computer systemcomprising: a display; at least one storage device; at least one networkcommunication device; at least one user input device; at least oneprocessor to execute instructions to cause the at least one processorto: cause presentation of a graphical user interface on the display toenable a user to build a graphical representation of a flow for anInternet of Things (IoT) system including a first electronic device anda second electronic device; cause, based on one or more user inputs, afirst block in the graphical representation of the flow to be presentedon the display, the first block associated with a first device type, thefirst device type associated with a first capability; associate, basedon the one or more user inputs, the first electronic device to implementthe first block for the first capability of the first device type in thegraphical user interface; cause, based on the one or more user inputs, asecond block in the graphical representation of the flow to be presentedon the display, the second block associated with a second device type,the second device type associated with a second capability, the secondblock to be positioned relative to the first block in the flow inresponse to a drag-and-drop input from the at least one user inputdevice; associate, based on the one or more user inputs, the secondelectronic device to implement the second block for the secondcapability of the second device type in the graphical user interface;cause, based on the one or more user inputs, a line to be presentedbetween the first and second blocks in the graphical representation ofthe flow, the line indicative of a relationship or interaction betweenthe first electronic device associated with the first block and thesecond electronic device associated with the second block; cause, basedon the one or more user inputs, a third block in the graphicalrepresentation of the flow to be presented on the display, the thirdblock associated with a log, wherein the flow for the IoT system is tocause state information for the second electronic device to be stored inthe log; and cause deployment of the flow to the IoT system, the secondcapability of the second device type to enable a user to interact withthe second electronic device to impact operation of the first electronicdevice after deployment based on the relationship or interaction betweenthe first and second electronic devices, the interaction to activate thefirst electronic device based on a state of the second electronicdevice, the first capability based on a model of the first electronicdevice, the instructions to cause the at least one processor todetermine a first plurality of electronic devices based on the firstcapability, the first plurality of electronic devices available forselection for association to implement the first block, the firstplurality of electronic devices include the first electronic device, theinstructions to cause the at least one processor to determine a secondplurality of electronic devices based on the second capability, thesecond plurality of electronic devices available for selection forassociation to implement the second block, the second plurality ofelectronic devices including the second electronic device.
 15. Thecomputer system of claim 14, wherein the first capability is to enable adisplay of information from the IoT system via a screen of the firstelectronic device, the information based on the relationship orinteraction between the first and second electronic devices.
 16. Thecomputer system of claim 14, wherein the first capability is based on amodel of the first electronic device.
 17. The computer system of claim14, wherein the interaction is to activate the first electronic devicebased on a state of the second electronic device.
 18. The computersystem of claim 14, wherein the interaction is to activate the firstelectronic device based on a comparison of a state of the secondelectronic device to the at least one user input device.
 19. Thecomputer system of claim 14, wherein the first device type is an outputand the second device type is an input.
 20. The computer system of claim14, wherein the at least one processor, to cause the deployment of theflow, is to cause distribution of instructions corresponding to thegraphical representation to a computing device.
 21. The computer systemof claim 14, wherein the state information includes an indication of acurrent output of the first block.